fee7f9d093
## Summary Follow up to https://github.com/astral-sh/uv/pull/15563 Closes https://github.com/astral-sh/uv/issues/13485 This is a first-pass at adding support for conditional support for Git LFS between git sources, initial feedback welcome. e.g. ``` [tool.uv.sources] test-lfs-repo = { git = "https://github.com/zanieb/test-lfs-repo.git", lfs = true } ``` For context previously a user had to set `UV_GIT_LFS` to have uv fetch lfs objects on git sources. This env var was all or nothing, meaning you must always have it set to get consistent behavior and it applied to all git sources. If you fetched lfs objects at a revision and then turned off lfs (or vice versa), the git db, corresponding checkout lfs artifacts would not be updated properly. Similarly, when git source distributions were built, there would be no distinction between sources with lfs and without lfs. Hence, it could corrupt the git, sdist, and archive caches. In order to support some sources being LFS enabled and other not, this PR adds a stateful layer roughly similar to how `subdirectory` works but for `lfs` since the git database, the checkouts and the corresponding caching layers needed to be LFS aware (requested vs installed). The caches also had to isolated and treated entirely separate when handling LFS sources. Summary * Adds `lfs = true` or `lfs = false` to git sources in pyproject.toml * Added `lfs=true` query param / fragments to most relevant url structs (not parsed as user input) * In the case of uv add / uv tool, `--lfs` is supported instead * `UV_GIT_LFS` environment variable support is still functional for non-project entrypoints (e.g. uv pip) * `direct-url.json` now has an custom `git_lfs` entry under VcsInfo (note, this is not in the spec currently -- see caveats). * git database and checkouts have an different cache key as the sources should be treated effectively different for the same rev. * sdists cache also differ in the cache key of a built distribution if it was built using LFS enabled revisions to distinguish between non-LFS same revisions. This ensures the strong assumption for archive-v0 that an unpacked revision "doesn't change sources" stays valid. Caveats * `pylock.toml` import support has not been added via git_lfs=true, going through the spec it wasn't clear to me it's something we'd support outside of the env var (for now). * direct-url struct was modified by adding a non-standard `git_lfs` field under VcsInfo which may be undersirable although the PEP 610 does say `Additional fields that would be necessary to support such VCS SHOULD be prefixed with the VCS command name` which could be interpret this change as ok. * There will be a slight lockfile and cache churn for users that use `UV_GIT_LFS` as all git lockfile entries will get a `lfs=true` fragment. The cache version does not need an update, but LFS sources will get their own namespace under git-v0 and sdist-v9/git hence a cache-miss will occur once but this can be sufficient to label this as breaking for workflows always setting `UV_GIT_LFS`. ## Test Plan Some initial tests were added. More tests likely to follow as we reach consensus on a final approach. For IT test, we may want to move to use a repo under astral namespace in order to test lfs functionality. Manual testing was done for common pathological cases like killing LFS fetch mid-way, uninstalling LFS after installing an sdist with it and reinstalling, fetching LFS artifacts in different commits, etc. PSA: Please ignore the docker build failures as its related to depot OIDC issues. --------- Co-authored-by: Zanie Blue <contact@zanie.dev> Co-authored-by: konstin <konstin@mailbox.org>
Crates
uv-bench
Functionality for benchmarking uv.
uv-cache-key
Generic functionality for caching paths, URLs, and other resources across platforms.
uv-distribution-filename
Parse built distribution (wheel) and source distribution (sdist) filenames to extract structured metadata.
uv-distribution-types
Abstractions for representing built distributions (wheels) and source distributions (sdists), and the sources from which they can be downloaded.
uv-install-wheel-rs
Install built distributions (wheels) into a virtual environment.
uv-once-map
A waitmap-like concurrent hash map for executing tasks
exactly once.
uv-pep440-rs
Utilities for interacting with Python version numbers and specifiers.
uv-pep508-rs
Utilities for parsing and evaluating dependency specifiers, previously known as PEP 508.
uv-platform-tags
Functionality for parsing and inferring Python platform tags as per PEP 425.
uv-cli
Command-line interface for the uv package manager.
uv-build-frontend
A PEP 517-compatible build frontend for uv.
uv-cache
Functionality for caching Python packages and associated metadata.
uv-client
Client for interacting with PyPI-compatible HTTP APIs.
uv-dev
Development utilities for uv.
uv-dispatch
A centralized struct for resolving and building source distributions in isolated environments.
Implements the traits defined in uv-types.
uv-distribution
Client for interacting with built distributions (wheels) and source distributions (sdists). Capable of fetching metadata, distribution contents, etc.
uv-extract
Utilities for extracting files from archives.
uv-fs
Utilities for interacting with the filesystem.
uv-git
Functionality for interacting with Git repositories.
uv-installer
Functionality for installing Python packages into a virtual environment.
uv-python
Functionality for detecting and leveraging the current Python interpreter.
uv-normalize
Normalize package and extra names as per Python specifications.
uv-requirements
Utilities for reading package requirements from pyproject.toml and requirements.txt files.
uv-resolver
Functionality for resolving Python packages and their dependencies.
uv-shell
Utilities for detecting and manipulating shell environments.
uv-types
Shared traits for uv, to avoid circular dependencies.
uv-pypi-types
General-purpose type definitions for types used in PyPI-compatible APIs.
uv-virtualenv
A venv replacement to create virtual environments in Rust.
uv-warnings
User-facing warnings for uv.
uv-workspace
Workspace abstractions for uv.
uv-requirements-txt
Functionality for parsing requirements.txt files.