## Summary
After chatting with the PyTorch team, it looks like some number of
wheels were accidentally uploaded with
`no-cache,no-store,must-revalidate` due to
https://github.com/pytorch/pytorch/pull/149218. They're going to correct
this for the respective wheels. I've encouraged them to set an immutable
caching header for these files, and it might happen. But even if this
isn't set, by default we only allow these wheels to be cached for 600s,
since the other wheels don't include a `Cache-Control` header at all
(but do include a `Last-Modified`, so we cache based on our heuristic:
`Freshness lifetime heuristically assumed because of presence of
last-modified header: 600s`). This probably leads to tons of unnecessary
downloads for users over time. Andrey from the PyTorch team agreed that
we should do this.
Closes https://github.com/astral-sh/uv/issues/15480.
## Summary
With this PR, we track the settings that were used to build a wheel
(`--config-settings`, plus any `extra-build-dependencies` or
`extra-build-variables`) and write those to the `.dist-info` directory
upon install. This then allows us to "reject" already-installed wheels,
if the user changes the build dependencies or `--config-settings` (or,
crucially, if they use `match-runtime = true` and the resolution
changes).
Closes https://github.com/astral-sh/uv/issues/15218.
## Summary
This breaks up a cycle I'm running into in incorporating the build
configuration into our cache keys. This is actually a type that ends up
in the frontend build system, etc., so I think it makes more sense here
anyway (as opposed to `uv-configuration` which tend to be our own
user-facing types).
## Summary
Make the use of `Self` consistent. Mostly done by running `cargo clippy
--fix -- -A clippy::all -W clippy::use_self`.
## Test Plan
<!-- How was it tested? -->
No need.
## Summary
This is an alternative to https://github.com/astral-sh/uv/pull/14944
that functions a little differently. Rather than adding separate
strategies, you can instead say:
```toml
[tool.uv.extra-build-dependencies]
child = [{ requirement = "anyio", match-runtime = true }]
```
Which will then enforce that `anyio` uses the same version as in the
lockfile.
## Summary
I should've noticed this during review -- my bad -- but it looks like
after lowering, we're converting back to `uv_pep508::Requirement`. This
is mostly okay, but it's lossy for some lowerings. For example, we lose
index pinning. With this PR, we now preserve the lowered types
(`Requirement`).
Closes https://github.com/astral-sh/uv/issues/15037.
`Candidate` has an optional field `prioritized`, which was mostly
redundant with `CandidateDist`. Specifically, it was only `None`, if
`CandidateDist` was `Installed`. This commit removes this duplication.
## Summary
The core problem here is that `allowed_indexes` only includes at most
one "default" index. This is problematic for tool upgrades, since the
index in the receipt will be marked as default, but credentials will be
omitted; if credentials are then defined in a `uv.toml`, we'll never
look at those, since that will _also_ be marked as default, and we only
look at the first default.
Instead, we should consider all defined indexes in priority order.
Closes https://github.com/astral-sh/uv/issues/14806.
## Summary
Right now, we write index URLs to the tool receipt with redacted
credentials (i.e., a username, and `****` in lieu of a password). This
is always wrong and unusable. Instead, this PR drops them entirely.
Part of https://github.com/astral-sh/uv/issues/14806.
## Summary
This was just an oversight on my part in the initial implementation.
Closes https://github.com/astral-sh/uv/issues/14719.
## Test Plan
With:
```toml
[project]
name = "foo"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13.2"
dependencies = [
]
[[tool.uv.index]]
url = "https://download.pytorch.org/whl/cpu"
cache-control = { api = "max-age=600" }
```
Ran `cargo run lock -vvv` and verified that the PyTorch index response
was cached (whereas it typically returns `cache-control:
no-cache,no-store,must-revalidate`).
Reviewing #14687, I noticed that we had implemented a
`Url::from_url_or_path`-like function, but it wasn't reusable. This
change `Verbatim::from_url_or_path` so we can use it in other places
too.
The PEP 508 parser is an odd place for this, but that's where
`VerbatimUrl` and `Scheme` are already living.
If a user specifies `-e /path/to/dir` and `/path/to/dir` in a `uv pip
install` command, we want the editable to "win" (rather than erroring
due to conflicting URLs). Unfortunately, this behavior meant that when
you requested a package as editable and non-editable in conflicting
groups, the editable version was _always_ used. This PR modifies the
requisite types to use `Option<bool>` rather than `bool` for the
`editable` field, so we can determine whether a requirement was
explicitly requested as editable, explicitly requested as non-editable,
or not specified (as in the case of `/path/to/dir` in a
`requirements.txt` file). In the latter case, we allow editables to
override the "unspecified" requirement.
If a project includes a path dependency twice, once with `editable =
true` and once without any `editable` annotation, those are now
considered conflicting URLs, and lead to an error, so I've marked this
change as breaking.
Closes https://github.com/astral-sh/uv/issues/14139.
## Summary
You can now override the cache control headers for the Simple API, file
downloads, or both:
```toml
[[tool.uv.index]]
name = "example"
url = "https://example.com/simple"
cache-control = { api = "max-age=600", files = "max-age=365000000, immutable" }
```
Closes https://github.com/astral-sh/uv/issues/10444.
Reverts:
- #14349
- #14346
- #14245
Retains the test cases. Includes a `find-links` test case.
Supersedes
- https://github.com/astral-sh/uv/pull/14387
- https://github.com/astral-sh/uv/pull/14503
We originally got a report at
https://github.com/astral-sh/uv/issues/13707 that inclusion of a
trailing slash on an index URL was causing lockfile churn despite having
no semantic meaning and resolved the issue by adding normalization that
stripped trailing slashes at parse time.
We then discovered that, while there are not semantic differences for
trailing slashes on Simple API index URLs, there are differences for
some flat (or find links) indexes. As reported in
https://github.com/astral-sh/uv/issues/14367, the change in
https://github.com/astral-sh/uv/pull/14245 caused a regression for at
least one user.
We attempted to fix the regression via a few approaches.
https://github.com/astral-sh/uv/pull/14387 attempted to differentiate
between Simple API and flat index URL parsing, but failed to account for
the `Deserialize` implementation, which always assumed Simple API-style
index URLs and incorrectly trimmed trailing slashes in various cases
where we deserialized the `IndexUrl` type from a file. I attempted to
resolve this by performing a larger refactor, but it ended up being
quite painful. In particular, the `Index` type was a blocker — we don't
know the `IndexUrl` variant until we've parsed the `IndexFormat` and
having a multi-stage deserializer is not appealing but adding a new
intermediate type (i.e., `RawIndex`) is painful due to the pervasiveness
of `Index`. Given that we've regressed behavior here and there's not a
straight-forward fix, we're reverting the normalization entirely.
https://github.com/astral-sh/uv/pull/14503 attempted to perform
normalization at compare-time, but that means we'd fail to invalidate
the lockfile when the a trailing slash was added or removed and given
that a trailing slash has semantic meaning for a find-links URL... we'd
have another correctness problem.
After this revert, we'll retain all index URLs verbatim. The downside to
this approach is that we'll be adding a bunch of trailing slashes back
to lockfiles that we previously normalized out, and we'll be reverting
our fix for users with inconsistent trailing slashes on their index
URLs. Users affected by the original motivating issue should use
consistent trailing slashes on their URLs, as they do frequently have
semantic meaning. We may want to revisit normalization and type aware
index URL parsing as part of a larger change.
Closes https://github.com/astral-sh/uv/issues/14367
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
We are using UV as a library and `installer()` returned `"pip\n"`. The
packages got installed by the pip package manager and not by UV. pip
seems to add a new line to the `INSTALLER` file and UV does not.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
The marker display code assumes that all versions are normalized, in
that all trailing zeroes are stripped. This is not the case for
tilde-equals and equals-star versions, where the trailing zeroes (before
the `.*`) are semantically relevant. This would cause path
dependent-behavior where we would get a different marker string
depending on whether a version with or without a trailing zero was added
to the cache first.
To handle both equals-star and tilde-equals when converting
`python_version` to `python_full_version` markers, we have to merge the
version normalization (i.e. trimming the trailing zeroes) and the
conversion both to `python_full_version` and to `Ranges`, while special
casing equals-star and tilde-equals.
To avoid churn in lockfiles, we only trim in the conversion to `Ranges`
for markers, but keep using untrimmed versions for requires-python.
(Note that this behavior is technically also path dependent, as versions
with and without trailing zeroes have the same Hash and Eq. E.q.,
`requires-python == ">= 3.10.0"` and `requires-python == ">= 3.10"` in
the same workspace could lead to either value in `uv.lock`, and which
one it is could change if we make unrelated (performance) changes.
Always trimming however definitely changes lockfiles, a churn I wouldn't
do outside another breaking or lockfile-changing change.) Nevertheless,
there is a change for users who have `requires-python = "~= 3.12.0"` in
their `pyproject.toml`, as this now hits the correct normalization path.
Fixes#14231Fixes#14270
## Summary
In #14245, we started normalizing index URLs by dropping the trailing
slash in the lockfile. We added tests to ensure that this didn't cause
existing lockfiles to be invalidated, but we missed one of the
constructors (specifically, the path that's used with
`tool.uv.sources`).
Close#7426
## Summary
Picking up on #8284, I noticed that the `requires_python` object already
has its specifiers canonicalized in the `intersection` method, meaning
`~=3.12` is converted to `>=3.12, <4`. To fix this, we check and warn in
`intersection`.
## Test Plan
Used the same tests from #8284.
This PR updates `IndexUrl` parsing to normalize non-file URLs by
removing trailing slashes. It also normalizes registry source URLs when
using them to validate the lockfile.
Prior to this change, when writing an index URL to the lockfile, uv
would use a trailing slash if present in the provided URL and no
trailing slash otherwise. This can cause surprising behavior. For
example, `uv lock --locked` will fail when a package is added with an
`--index` value without a trailing slash and then `uv lock --locked` is
run with a `pyproject.toml` version of the index URL that contains a
trailing slash. This PR fixes this and adds a test for the scenario.
It might be safe to normalize file URLs in the same way, but since
slashes have a well-defined meaning in the context of files and
directories, I chose not to normalize them here.
Closes#13707.
We do not currently support passing index names to `--index` for
installing packages. However, we do accept relative paths that can look
like index names. This PR adds the requirement that `--index` values
must be disambiguated with a prefix (`./` or `../` on Unix and Windows
or `.\\` or `..\\` on Windows). For now, if an ambiguous value is
provided, uv will warn that this will not be supported in the future.
Currently, if you provide an index name like `--index test` when there
is no `test` directory, uv will error with a `Directory not found...`
error. That's not very informative if you thought index names were
supported. The new warning makes the context clearer.
Closes#13921
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Update [schemars
0.9.0](https://github.com/GREsau/schemars/releases/tag/v0.9.0)
There are differences in the generated JSON Schema and I will [contact
the author](https://github.com/GREsau/schemars/issues/407).
## Test Plan
---------
Co-authored-by: konstin <konstin@mailbox.org>
Close#13922
## Summary
Add a warning if the directory given by the `--index` argument is empty.
## Test Plan
Added test case `add_index_empty_directory` in `edit.rs`
(or legacy tool.uv.workspace).
This cleaves out a dedicated SourcedDependencyGroups type based on
RequiresDist but with only the DependencyGroup handling implemented.
This allows `uv pip` to read `dependency-groups` from pyproject.tomls
that only have that table defined, per PEP 735, and as implemented by
`pip`.
However we want our implementation to respect various uv features when
they're available:
* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)
As such we want to opportunistically detect "as much as possible" while
doing as little as possible when things are missing. The issue with the
old RequiresDist path was that it fundamentally wanted to build the
package, and if `[project]` was missing it would try to desperately run
setuptools on the pyproject.toml to try to find metadata and make a hash
of things.
At the same time, the old code also put in a lot of effort to try to
pretend that `uv pip` dependency-groups worked like `uv`
dependency-groups with defaults and non-only semantics, only to separate
them back out again. By explicitly separating them out, we confidently
get the expected behaviour.
Note that dependency-group support is still included in RequiresDist, as
some `uv` paths still use it. It's unclear to me if those paths want
this same treatment -- for now I conclude no.
Fixes#13138
This allows you to specify requires-python on individual dependency-groups,
with the intended usecase being "oh my dev-dependencies have a higher
requires-python than my actual project".
This includes a large driveby move of the RequiresPython type to
uv-distribution-types to allow us to generate the appropriate markers at
this point in the code. It also migrates RequiresPython from
pubgrub::Range to version_ranges::Ranges, and makes several pub(crate)
items pub, as it's no longer defined in uv_resolver.
Fixes#11606
Prior to this PR, there were numerous places where uv would leak
credentials in logs. We had a way to mask credentials by calling methods
or a recently-added `redact_url` function, but this was not secure by
default. There were a number of other types (like `GitUrl`) that would
leak credentials on display.
This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials
when logging by default. It takes a maximalist approach, replacing the
use of `Url` almost everywhere. This includes when first parsing config
files, when storing URLs in types like `GitUrl`, and also when storing
URLs in types that in practice will never contain credentials (like
`DirectorySourceUrl`). The idea is to make it easy for developers to do
the right thing and for the compiler to support this (and to minimize
ever having to manually convert back and forth). Displaying credentials
now requires an active step. Note that despite this maximalist approach,
the use of the newtype should be zero cost.
One conspicuous place this PR does not use `DisplaySafeUrl` is in the
`uv-auth` crate. That would require new clones since there are calls to
`request.url()` that return a `&Url`. One option would have been to make
`DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime
annotations all over the codebase. I've created a separate PR based on
this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with
one new clone. We can discuss the tradeoffs there.
Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is
`uv_redacted/lib.rs`, where the newtype is implemented. To make it
easier to review the rest, here are some points of note:
* `DisplaySafeUrl` has a `Display` implementation that masks
credentials. Currently, it will still display the username when there is
both a username and password. If we think is the wrong choice, it can
now be changed in one place.
* `DisplaySafeUrl` has a `remove_credentials()` method and also a
`.to_string_with_credentials()` method. This allows us to use it in a
variety of scenarios.
* `IndexUrl::redacted()` was renamed to
`IndexUrl::removed_credentials()` to make it clearer that we are not
masking.
* We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest`
methods like `.get()` and `.head()`.
* We convert from a `DisplaySafeUrl` to a `Url` when creating a
`uv_auth::Index`. That is because, as mentioned above, I will be
updating the `uv_auth` crate to use this newtype in a separate PR.
* A number of tests (e.g., in `pip_install.rs`) that formerly used
filters to mask tokens in the test output no longer need those filters
since tokens in URLs are now masked automatically.
* The one place we are still knowingly writing credentials to
`pyproject.toml` is when a URL with credentials is passed to `uv add`
with `--raw`. Since displaying credentials is no longer automatic, I
have added a `to_string_with_credentials()` method to the `Pep508Url`
trait. This is used when `--raw` is passed. Adding it to that trait is a
bit weird, but it's the simplest way to achieve the goal. I'm open to
suggestions on how to improve this, but note that because of the way
we're using generic bounds, it's not as simple as just creating a
separate trait for that method.
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Fixes#11970.
## Test Plan
<!-- How was it tested? -->
Ran `cargo nextest`
uv’s default index strategy was designed with dependency confusion
attacks in mind. [According to the
docs](https://docs.astral.sh/uv/configuration/indexes/#searching-across-multiple-indexes),
“if a package exists on an internal index, it should always be installed
from the internal index, and never from PyPI”. Unfortunately, this is
not true in the case where authentication fails on that internal index.
In that case, uv will simply try the next index (even on the
`first-index` strategy). This means that uv is not secure by default in
this common scenario.
This PR causes uv to stop searching for a package if it encounters an
authentication failure at an index. It is possible to opt out of this
behavior for an index with a new `pyproject.toml` option
`ignore-error-codes`. For example:
```
[[tool.uv.index]]
name = "my-index"
url = "<index-url>"
ignore-error-codes = [401, 403]
```
This will also enable users to handle idiosyncratic registries in a more
fine-grained way. For example, PyTorch registries return a 403 when a
package is not found. In this PR, we special-case PyTorch registries to
ignore 403s, but users can use `ignore-error-codes` to handle similar
behaviors if they encounter them on internal registries.
Depends on #12651Closes#9429Closes#12362
Charlie Marsh
samypr100
adamnemecek
Zanie Blue
konsti
Nils Koch
John Mumm
Aaron Ang
ya7010
Frazer McLean
Aria Desires
Christopher Tee