## Summary
Follow up to https://github.com/astral-sh/uv/pull/15563
Closes https://github.com/astral-sh/uv/issues/13485
This is a first-pass at adding support for conditional support for Git
LFS between git sources, initial feedback welcome.
e.g.
```
[tool.uv.sources]
test-lfs-repo = { git = "https://github.com/zanieb/test-lfs-repo.git", lfs = true }
```
For context previously a user had to set `UV_GIT_LFS` to have uv fetch
lfs objects on git sources. This env var was all or nothing, meaning you
must always have it set to get consistent behavior and it applied to all
git sources. If you fetched lfs objects at a revision and then turned
off lfs (or vice versa), the git db, corresponding checkout lfs
artifacts would not be updated properly. Similarly, when git source
distributions were built, there would be no distinction between sources
with lfs and without lfs. Hence, it could corrupt the git, sdist, and
archive caches.
In order to support some sources being LFS enabled and other not, this
PR adds a stateful layer roughly similar to how `subdirectory` works but
for `lfs` since the git database, the checkouts and the corresponding
caching layers needed to be LFS aware (requested vs installed). The
caches also had to isolated and treated entirely separate when handling
LFS sources.
Summary
* Adds `lfs = true` or `lfs = false` to git sources in pyproject.toml
* Added `lfs=true` query param / fragments to most relevant url structs
(not parsed as user input)
* In the case of uv add / uv tool, `--lfs` is supported instead
* `UV_GIT_LFS` environment variable support is still functional for
non-project entrypoints (e.g. uv pip)
* `direct-url.json` now has an custom `git_lfs` entry under VcsInfo
(note, this is not in the spec currently -- see caveats).
* git database and checkouts have an different cache key as the sources
should be treated effectively different for the same rev.
* sdists cache also differ in the cache key of a built distribution if
it was built using LFS enabled revisions to distinguish between non-LFS
same revisions. This ensures the strong assumption for archive-v0 that
an unpacked revision "doesn't change sources" stays valid.
Caveats
* `pylock.toml` import support has not been added via git_lfs=true,
going through the spec it wasn't clear to me it's something we'd support
outside of the env var (for now).
* direct-url struct was modified by adding a non-standard `git_lfs`
field under VcsInfo which may be undersirable although the PEP 610 does
say `Additional fields that would be necessary to support such VCS
SHOULD be prefixed with the VCS command name` which could be interpret
this change as ok.
* There will be a slight lockfile and cache churn for users that use
`UV_GIT_LFS` as all git lockfile entries will get a `lfs=true` fragment.
The cache version does not need an update, but LFS sources will get
their own namespace under git-v0 and sdist-v9/git hence a cache-miss
will occur once but this can be sufficient to label this as breaking for
workflows always setting `UV_GIT_LFS`.
## Test Plan
Some initial tests were added. More tests likely to follow as we reach
consensus on a final approach.
For IT test, we may want to move to use a repo under astral namespace in
order to test lfs functionality.
Manual testing was done for common pathological cases like killing LFS
fetch mid-way, uninstalling LFS after installing an sdist with it and
reinstalling, fetching LFS artifacts in different commits, etc.
PSA: Please ignore the docker build failures as its related to depot
OIDC issues.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: konstin <konstin@mailbox.org>
Part of https://github.com/astral-sh/uv/issues/4392
We shouldn't link to PyPI, and dropping the workspace-level
documentation link should mean that we get the auto-generated `docs.rs`
links.
Resolves https://github.com/astral-sh/uv/issues/16719
`uv add` collapses multiple spaces before inline comments in
`[project.dependencies]`, causing unrelated diffs and moving comments
onto the wrong columns. This diff captures the exact whitespace padding
that preceded each end-of-line comment when parsing the array and reuses
it when formatting.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
This PR adds an `exclude-dependencies` setting that allows users to omit
a dependency during resolution. It's effectively a formalized version of
the `flask ; python_version < '0'` hack that we've suggested to users in
various issues.
Closes#12616.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes#16285
## Test Plan
<!-- How was it tested? -->
## Summary
Fixes [1633](https://github.com/astral-sh/uv/issues/16133). Preserves
comments preceding "version = ..." line when uv version --bump is ran
## Test Plan
Added IT test
---------
Co-authored-by: konsti <konstin@mailbox.org>
## Summary
This ended up being a bit more complex, similar to `package = false`,
because we need to understand the editable status _globally_ across the
workspace based on the packages that depend on it.
Closes https://github.com/astral-sh/uv/issues/15686.
## Summary
This was fixed in https://github.com/astral-sh/uv/pull/15161, then
reverted as it regressed the error handling. I've re-applied the change
here, but moved the error handling to the runtime, rather than
parse-time. I think this is slightly worse in that we no longer include
the originating source code snippet, but it at least gives us the
expected behavior :(
Closes https://github.com/astral-sh/uv/issues/15124.
## Summary
`match-runtime` can be explicitly specified, and if it's `false` it
should behave the same way as if it's omitted.
## Test Plan
Added snapshot test
## Summary
It would be nice if this rendered as
`[tool.uv.extra-build-dependencies]` and `[extra-build-dependencies]`
(in `uv.toml`), but this is at least correct.
Closes https://github.com/astral-sh/uv/issues/15124.
## Summary
Make the use of `Self` consistent. Mostly done by running `cargo clippy
--fix -- -A clippy::all -W clippy::use_self`.
## Test Plan
<!-- How was it tested? -->
No need.
## Summary
This is an alternative to https://github.com/astral-sh/uv/pull/14944
that functions a little differently. Rather than adding separate
strategies, you can instead say:
```toml
[tool.uv.extra-build-dependencies]
child = [{ requirement = "anyio", match-runtime = true }]
```
Which will then enforce that `anyio` uses the same version as in the
lockfile.
## Summary
The basic problem here is that when we had multiple items in an inline
array, and that array expanded to multiple lines, we accidentally
changed the indentation part-way through due to how prefixes work in the
TOML.
Here's Claude's explanation of the root cause, which I find pretty
decent:
```
Here's what happened step by step:
1. First item ("iniconfig"): Has empty prefix "" → indentation_prefix stays None → uses default 4 spaces
2. Second item ("ruff"): Has empty prefix "" → indentation_prefix stays None → uses default 4 spaces
3. Third item ("typing-extensions"): Has prefix " " (single space from inline format) → indentation_prefix becomes
Some(" ") → uses only 1 space!
This produced:
[dependency-groups]
dev = [
"iniconfig>=2.0.0",
"ruff",
"typing-extensions", # ← Only 1 space instead of 4!
]
Why the Third Item Had a Different Prefix
In inline arrays like ["ruff", "typing-extensions"], the items are separated by commas and spaces. When parsed by
the TOML library:
- "ruff" has no prefix (it comes right after [)
- "typing-extensions" has a single space prefix (the space after the comma)
The Fix
Moving the indentation calculation outside the loop ensures it's calculated only once:
// Calculate indentation ONCE before the loop
if let Some(first_item) = deps.iter().next() {
let decor_prefix = /* get prefix from first item */
indentation_prefix = (!decor_prefix.is_empty()).then_some(decor_prefix.to_string());
}
// Now use the same indentation for ALL items
for item in deps.iter_mut() {
// Apply consistent indentation to every item
}
This ensures all items get the same indentation (4 spaces by default when converting from inline arrays), producing
the correct output:
[dependency-groups]
dev = [
"iniconfig>=2.0.0",
"ruff",
"typing-extensions", # ← Correct 4-space indentation
]
The bug only affected arrays being converted from inline to multiline format, where different items might have
different residual formatting from their inline representation.
```
Closes#14961.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Replaces https://github.com/astral-sh/uv/pull/14092
Adds `tool.uv.extra-build-dependencies = {package = [dependency, ...]}`
which extends `build-system.requires` during package builds.
These are lowered via workspace sources, are applied to transitive
dependencies, and are included in the wheel cache shard hash.
There are some features we need to follow-up on, but are out of scope
here:
- Preferring locked versions for build dependencies
- Settings for requiring locked versions for build depencies
There are some quality of life follow-ups we should also do:
- Warn on `extra-build-dependencies` that do not apply to any packages
- Add test cases and improve error messaging when the
`extra-build-dependencies` resolve fails
-------
There ~are~ were a few open decisions to be made here
1. Should we resolve these dependencies alongside the
`build-system.requires` dependencies? Or should we resolve separately?
(I think the latter is more powerful? because you can override things?
but it opens the door to breaking your build)
2. Should we install these dependencies into the same environment? Or
should we layer it on top as we do elsewhere? (I think it's fine to
install into the same environment)
3. Should we respect sources defined in the parent project? (I think
yes, but then we need to lower the dependencies earlier — I don't think
that's a big deal, but it's not implemented)
4. Should we respect sources defined in the child project? (I think no,
this gets really complicated and seems weird to allow)
5. Should we apply this to transitive dependencies? (I think so)
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
A little nuanced, but... When you add multiple `--index` URLs on the CLI
(e.g., in `uv pip install`), we check the first-provided index, then the
second index, etc. However, when we _write_ those URLs to the
`pyproject.toml` in `uv add`, we were adding them in reverse-order. We
now add them in a way that preserves the priority order.
Closes https://github.com/astral-sh/uv/issues/14817.
## Summary
This should give us some performance and error message improvements.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
We currently treat path sources as virtual if they do not specify a
build system, which is surprising behavior. This PR updates the behavior
to treat path sources as packages unless the path source is explicitly
marked as `package = false` or its own `tool.uv.package` is set to
`false`.
Closes#12015
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
If a user specifies `-e /path/to/dir` and `/path/to/dir` in a `uv pip
install` command, we want the editable to "win" (rather than erroring
due to conflicting URLs). Unfortunately, this behavior meant that when
you requested a package as editable and non-editable in conflicting
groups, the editable version was _always_ used. This PR modifies the
requisite types to use `Option<bool>` rather than `bool` for the
`editable` field, so we can determine whether a requirement was
explicitly requested as editable, explicitly requested as non-editable,
or not specified (as in the case of `/path/to/dir` in a
`requirements.txt` file). In the latter case, we allow editables to
override the "unspecified" requirement.
If a project includes a path dependency twice, once with `editable =
true` and once without any `editable` annotation, those are now
considered conflicting URLs, and lead to an error, so I've marked this
change as breaking.
Closes https://github.com/astral-sh/uv/issues/14139.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
remove redundant words in comment
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
Signed-off-by: jingchanglu <jingchanglu@outlook.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Update [schemars
0.9.0](https://github.com/GREsau/schemars/releases/tag/v0.9.0)
There are differences in the generated JSON Schema and I will [contact
the author](https://github.com/GREsau/schemars/issues/407).
## Test Plan
---------
Co-authored-by: konstin <konstin@mailbox.org>
(or legacy tool.uv.workspace).
This cleaves out a dedicated SourcedDependencyGroups type based on
RequiresDist but with only the DependencyGroup handling implemented.
This allows `uv pip` to read `dependency-groups` from pyproject.tomls
that only have that table defined, per PEP 735, and as implemented by
`pip`.
However we want our implementation to respect various uv features when
they're available:
* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)
As such we want to opportunistically detect "as much as possible" while
doing as little as possible when things are missing. The issue with the
old RequiresDist path was that it fundamentally wanted to build the
package, and if `[project]` was missing it would try to desperately run
setuptools on the pyproject.toml to try to find metadata and make a hash
of things.
At the same time, the old code also put in a lot of effort to try to
pretend that `uv pip` dependency-groups worked like `uv`
dependency-groups with defaults and non-only semantics, only to separate
them back out again. By explicitly separating them out, we confidently
get the expected behaviour.
Note that dependency-group support is still included in RequiresDist, as
some `uv` paths still use it. It's unclear to me if those paths want
this same treatment -- for now I conclude no.
Fixes#13138
This allows you to specify requires-python on individual dependency-groups,
with the intended usecase being "oh my dev-dependencies have a higher
requires-python than my actual project".
This includes a large driveby move of the RequiresPython type to
uv-distribution-types to allow us to generate the appropriate markers at
this point in the code. It also migrates RequiresPython from
pubgrub::Range to version_ranges::Ranges, and makes several pub(crate)
items pub, as it's no longer defined in uv_resolver.
Fixes#11606
By default, uv uses only a lower bound in `uv add`, which avoids
dependency conflicts due to upper bounds. With this PR, this cna be
changed by setting a different bound kind. The bound kind can be
configured in `uv.toml`, as a user preference, in `pyproject.toml`, as a
project preference, or on the CLI, when adding a specific project.
We add two options that add an upper bound on the constraint, one for
SemVer (`>=1.2.3,<2.0.0`, dubbed "major", modeled after the SemVer
caret) and another one for dependencies that make breaking changes in
minor version (`>=1.2.3,<1.3.0`, dubbed "minor", modeled after the
SemVer tilde). Intuitively, the major option bumps the most significant
version component, while the minor option bumps the second most
significant version component. There is also an exact bounds option
(`==1.2.3`), though generally we recommend setting a wider bound and
using the lockfile for pinning.
Versions can have leading zeroes, such as `0.1` or `0.0.1`. For a single
leading 0, we shift the the meaning of major and minor similar to cargo.
For two or more leading zeroes, the difference between major and minor
becomes inapplicable, instead both bump the most significant component:
- major: `0.1` -> `>=0.1,<0.2`
- major: `0.0.1` -> `>=0.0.1,<0.0.2`
- major: `0.0.1.1` -> `>=0.0.1.1,<0.0.2.0`
- major: `0.0.0.1` -> `>=0.0.0.1,<0.0.0.2`
- minor: `0.1` -> `>=0.1,<0.1.1`
- minor: `0.0.1` -> `>=0.0.1,<0.0.2`
- minor: `0.0.1.1` -> `>=0.0.1.1,<0.0.2.0`
- minor: `0.0.0.1` -> `>=0.0.0.1,<0.0.0.2`
For a consistent appearance, we try to preserve the number of components
in the upper bound. For example, adding a version `2.17` with the major
option is stored as `>=2.17,<3.0`. If a version uses three components
and is greater than 0, both bounds will also use three components
(SemVer versions always have three components). Of the top 100 PyPI
packages, 8 use a non-three-component version (docutils, idna, pycparser
and soupsieve with two components, packaging, pytz and tzdata with two
component, CalVer and trove-classifiers with four component CalVer).
Example `pyproject.toml` files with the top 100 packages: [`--bounds
major`](https://gist.github.com/konstin/0aaffa9ea53c4834c22759e8865409f4)
and [`--bounds
minor`](https://gist.github.com/konstin/e77f5e990a7efe8a3c8a97c5c5b76964).
While many projects follow version scheme that roughly or directly
matches the major or minor options, these compatibility ranges are
usually not applicable for the also popular CalVer versioning.
For pre-release versions, there are two framings we could take: One is
that pre-releases generally make no guarantees about compatibility
between them and are used to introduce breaking changes, so we should
pin them exactly. In many cases however, pre-release specifiers are used
because a project needs a bugfix or a feature that hasn't made it into a
stable release, or because a project is compatible with the next version
before a final version for that release is published. In those cases,
compatibility with other packages that depend on the same library is
more important, so the desired bound is the same as it would be for the
stable release, except with the lower bound lowered to include
pre-release.
The names of the bounds and the name of the flag is up for bikeshedding.
Currently, the option is call `tool.uv.bounds`, but we could also move
it under `tool.uv.edit.bounds`, where it would be the first/only entry.
Fixes#6783
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Prior to this PR, there were numerous places where uv would leak
credentials in logs. We had a way to mask credentials by calling methods
or a recently-added `redact_url` function, but this was not secure by
default. There were a number of other types (like `GitUrl`) that would
leak credentials on display.
This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials
when logging by default. It takes a maximalist approach, replacing the
use of `Url` almost everywhere. This includes when first parsing config
files, when storing URLs in types like `GitUrl`, and also when storing
URLs in types that in practice will never contain credentials (like
`DirectorySourceUrl`). The idea is to make it easy for developers to do
the right thing and for the compiler to support this (and to minimize
ever having to manually convert back and forth). Displaying credentials
now requires an active step. Note that despite this maximalist approach,
the use of the newtype should be zero cost.
One conspicuous place this PR does not use `DisplaySafeUrl` is in the
`uv-auth` crate. That would require new clones since there are calls to
`request.url()` that return a `&Url`. One option would have been to make
`DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime
annotations all over the codebase. I've created a separate PR based on
this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with
one new clone. We can discuss the tradeoffs there.
Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is
`uv_redacted/lib.rs`, where the newtype is implemented. To make it
easier to review the rest, here are some points of note:
* `DisplaySafeUrl` has a `Display` implementation that masks
credentials. Currently, it will still display the username when there is
both a username and password. If we think is the wrong choice, it can
now be changed in one place.
* `DisplaySafeUrl` has a `remove_credentials()` method and also a
`.to_string_with_credentials()` method. This allows us to use it in a
variety of scenarios.
* `IndexUrl::redacted()` was renamed to
`IndexUrl::removed_credentials()` to make it clearer that we are not
masking.
* We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest`
methods like `.get()` and `.head()`.
* We convert from a `DisplaySafeUrl` to a `Url` when creating a
`uv_auth::Index`. That is because, as mentioned above, I will be
updating the `uv_auth` crate to use this newtype in a separate PR.
* A number of tests (e.g., in `pip_install.rs`) that formerly used
filters to mask tokens in the test output no longer need those filters
since tokens in URLs are now masked automatically.
* The one place we are still knowingly writing credentials to
`pyproject.toml` is when a URL with credentials is passed to `uv add`
with `--raw`. Since displaying credentials is no longer automatic, I
have added a `to_string_with_credentials()` method to the `Pep508Url`
trait. This is used when `--raw` is passed. Adding it to that trait is a
bit weird, but it's the simplest way to achieve the goal. I'm open to
suggestions on how to improve this, but note that because of the way
we're using generic bounds, it's not as simple as just creating a
separate trait for that method.
Zanie Blue
samypr100
Zsolt Dollenstein
Charlie Marsh
liam
Mark Dodgson
Assad Yousuf
Aria Desires
konsti
Ahmed Ilyas
adamnemecek
Ibraheem Ahmed
John Mumm
荆长逯
ya7010