Hey, are you okay with exposing the `ErrorTree` for library consumers?
We have a use case that needs more information on conflicts. We need the
tree-structure of the conflict and be able to traverse it in particular.
Signed-off-by: Simon Sure <ssure@palantir.com>
## Summary
There's a good example of the downside of using verbatim URLs here:
https://github.com/astral-sh/uv/pull/14197#discussion_r2163599625 (we
show two relative paths that point to the same directory, but it's not
clear from the error message).
The diff:
```
2 2 │ ----- stdout -----
3 3 │
4 4 │ ----- stderr -----
5 5 │ error: Requirements contain conflicting URLs for package `library` in all marker environments:
6 │-- ../../library
7 │-- ./library
6 │+- file://[TEMP_DIR]/library
7 │+- file://[TEMP_DIR]/library (editable)
```
## Summary
When the user provides a requirement like `==2.4.*`, we desugar that to
`>=2.4.dev0,<2.5.dev0`. These bounds then appear in error messages, and
worse, they also trick the error message reporter into thinking that the
user asked for a pre-release.
This PR adds logic to convert to the more-concise `==2.4.*`
representation when possible. We could probably do a similar thing for
the compatible release operator (`~=`).
Closes https://github.com/astral-sh/uv/issues/14177.
Co-authored-by: Zanie Blue <contact@zanie.dev>
Users are not (yet) properly familiar with the concept of universal
resolution and its implication that we need to resolve for all possible
platforms and Python versions. Some projects only target a specific
platform or Python version, and users experience resolution errors due
to failures for other platforms. Indicated by the number of questions we
get about it, `tool.uv.environments` for restricting environments is not
well discoverable.
We add a special hint when resolution failed on a fork disjoint with the
current environment, hinting the user to constrain `requires-python` and
`tool.uv.environments` respectively.
The hint has false positives for cases where the resolution failed on a
different platform, but equally fails on the current platform, in cases
where the non-current fork was tried earlier. Given that conflicts can
be based on `requires-python`, afaik we can't parse whether the current
platform would also be affected from the derivation tree.
Two cases not covered by this are build errors as well as install errors
that need `tool.uv.required-environments`.
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
When an index performs a bogus redirect or otherwise returns a different
distribution name than expected, uv currently hangs.
In the example case, requesting the simple index page for any package
returns the page for anyio. This mean querying the sniffio version map
returns only anyio entries, and the version maps resolves to an anyio
version. When the resolver makes a query for sniffio and waits for it to
resolve, the main thread finds an anyio and resolves only that in the
wait map, causing the hang.
We fix this by checking the name of the returned distribution against
the name of the requested distribution. For good measure, we add the
same check in `Request::Dist` and `Request::Installed`. For performance
and complexity reasons, we don't perform this check in the version map
itself, but only after a candidate distribution has been selected.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This removes the error that was causing folks problems.
This does result in some snapshot updates that are arguably wrong, or at
least sub-optimal. However, it's actually intended. Because the approach
we're going to take is going to permit the creation of uninstallable
lock files as a side effect. In the future, we will modify this test to
check that, while `uv lock` succeeds, `uv sync` will always fail.
## Summary
The idea here is to show both (1) an example of a compatible tag and (2)
the tags that were available, whenever we fail to resolve due to an
abscence of matching wheels.
Closes https://github.com/astral-sh/uv/issues/2777.
## Summary
If a user provides a constraint like `flask==3.0.0`, that gets expanded
to `[3.0.0, 3.0.0+[max])`. So it's not a _singleton_, but it should be
treated as such for the purposes of prioritization, since in practice it
will almost always map to a single version.
## Summary
This PR revives https://github.com/astral-sh/uv/pull/7827 to improve
tool resolutions such that, if the resolution fails, and the selected
interpreter doesn't match the required Python version from the solve, we
attempt to re-solve with a newly-discovered interpreter that _does_
match the required Python version.
For now, we attempt to choose a Python interpreter that's greater than
the inferred `requires-python`, but compatible with the same Python
minor. This helps avoid successive failures for cases like Posting,
where choosing Python 3.13 fails because it has a dependency that lacks
source distributions and doesn't publish any Python 3.13 wheels. We
should further improve the strategy to solve _that_ case too, but this
is at least the more conservative option...
In short, if you do `uv tool instal posting`, and we find Python 3.8 on
your machine, we'll detect that `requires-python: >=3.11`, then search
for the latest Python 3.11 interpreter and re-resolve.
Closes https://github.com/astral-sh/uv/issues/6381.
Closes https://github.com/astral-sh/uv/issues/10282.
## Test Plan
The following should succeed:
```
cargo run python uninstall --all
cargo run python install 3.8
cargo run tool install posting
```
In the logs, we see:
```
...
DEBUG No compatible version found for: posting
DEBUG Refining interpreter with: Python >=3.11, <3.12
DEBUG Searching for Python >=3.11, <3.12 in managed installations or search path
DEBUG Searching for managed installations at `/Users/crmarsh/.local/share/uv/python`
DEBUG Skipping incompatible managed installation `cpython-3.8.20-macos-aarch64-none`
DEBUG Found `cpython-3.13.1-macos-aarch64-none` at `/opt/homebrew/bin/python3` (search path)
DEBUG Skipping interpreter at `/opt/homebrew/opt/python@3.13/bin/python3.13` from search path: does not satisfy request `>=3.11, <3.12`
DEBUG Found `cpython-3.11.7-macos-aarch64-none` at `/opt/homebrew/bin/python3.11` (search path)
DEBUG Re-resolving with Python 3.11.7
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.7
DEBUG Solving with target Python version: >=3.11.7
DEBUG Adding direct dependency: posting*
DEBUG Searching for a compatible version of posting (*)
...
```
Closes https://github.com/astral-sh/uv/issues/9891
There are two changes here
1. We now exclude pre-releases (if they are not allowed) from the
available versions set when simplifying ranges, this means the
simplified range reflects the _allowed_ available versions — which is
what we want. We no longer segment ranges into arbitrary looking
segments..
2. We improve on #9885, expanding the scope to avoid regressions where
we would now otherwise enumerate a bunch of versions
---------
Co-authored-by: konsti <konstin@mailbox.org>
The resolver methods are already too large and complex, especially
`choose_version*`, so i wanted to shrink and simplify them a bit before
adding new methods to them.
I've split `MetadataResponse` into three variants: success, non-fatal
error (reported through pubgrub), fatal error (reported as error trace).
The resulting non-fatal `MetadataUnavailable` type is equivalent to the
`IncompletePackage` type, so they are now merged. (`UnavailableVersion`
is a bit different since, besides the extra `IncompatibleDist` variant,
it have no error source attached). This shows that the missing metadata
variant was unused, which I removed.
Tagging as error messages for the logging format changes.
Instead of modifying the error to replace a dummy derivation chain from
construction with the real one, build the error with the real derivation
chain directly.
This came up when trying to improve the build error reporting.
Introduces `DistErrorKind` to avoid error variants for each case that
are only different in one line of the message.
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
## Summary
This PR adds context to our error messages to explain _why_ a given
package was included, if we fail to download or build it.
It's quite a large change, but it motivated some good refactors and
improvements along the way.
Closes https://github.com/astral-sh/uv/issues/8962.
This PR adds support for conflicting extras. For example, consider
some optional dependencies like this:
```toml
[project.optional-dependencies]
project1 = ["numpy==1.26.3"]
project2 = ["numpy==1.26.4"]
```
These dependency specifications are not compatible with one another.
And if you ask uv to lock these, you'll get an unresolvable error.
With this PR, you can now add this to your `pyproject.toml` to get
around this:
```toml
[tool.uv]
conflicting-groups = [
[
{ package = "project", extra = "project1" },
{ package = "project", extra = "project2" },
],
]
```
This will make the universal resolver create additional forks
internally that keep the dependencies from the `project1` and
`project2` extras separate. And we make all of this work by reporting
an error at **install** time if one tries to install with two or more
extras that have been declared as conflicting. (If we didn't do this,
it would be possible to try and install two different versions of the
same package into the same environment.)
This PR does *not* add support for conflicting **groups**, but it is
intended to add support in a follow-up PR.
Closes#6981Fixes#8024
Ref #6729, Ref #6830
This should also hopefully unblock
https://github.com/dagster-io/dagster/pull/23814, but in my testing, I
did run into other problems (specifically, with `pywin`). But it does
resolve the problem with incompatible dependencies in two different
extras once you declare `test-airflow-1` and `test-airflow-2` as
conflicting for `dagster-airflow`.
NOTE: This PR doesn't make `conflicting-groups` public yet. And in a
follow-up PR, I plan to switch the name to `conflicts` instead of
`conflicting-groups`, since it will be able to accept conflicting extras
_and_ conflicting groups.
## Summary
We're inconsistent with these -- sometimes it's `Error::Fetch` and
sometimes it's `Error::Download`. The message says download, so let's
just use that?
Implement a full working version of local version semantics. The (AFAIA)
major move towards this was implemented in #2430. This added support
such that the version specifier `torch==2.1.0+cpu` would install
`torch@2.1.0+cpu` and consider `torch@2.1.0+cpu` a valid way to satisfy
the requirement `torch==2.1.0` in further dependency resolution.
In this feature, we more fully support local version semantics. Namely,
we now allow `torch==2.1.0` to install `torch@2.1.0+cpu` regardless of
whether `torch@2.1.0` (no local tag) actually exists.
We do this by adding an internal-only `Max` value to local versions that
compare greater to all other local versions. Then we can translate
`torch==2.1.0` into bounds: greater than 2.1.0 with no local tag and
less than 2.1.0 with the `Max` local tag.
Depends on https://github.com/astral-sh/packse/pull/227.
This updates the surrounding code to use the new ResolverEnvironment
type. In some cases, this simplifies caller code by removing case
analysis. There *shouldn't* be any behavior changes here. Some test
snapshots were updated to account for some minor tweaks to error
messages.
I didn't split this up into separate commits because it would have been
too difficult/costly.
## Summary
This PR lifts the restriction that a package must come from a single
index. For example, you can now do:
```toml
[project]
name = "project"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["jinja2"]
[tool.uv.sources]
jinja2 = [
{ index = "torch-cu118", marker = "sys_platform == 'darwin'"},
{ index = "torch-cu124", marker = "sys_platform != 'darwin'"},
]
[[tool.uv.index]]
name = "torch-cu118"
url = "https://download.pytorch.org/whl/cu118"
[[tool.uv.index]]
name = "torch-cu124"
url = "https://download.pytorch.org/whl/cu124"
```
The construction is very similar to the way we handle URLs today: you
can have multiple URLs for a given package, but they must appear in
disjoint forks. So most of the code is just adding that abstraction to
the resolver, following our handling of URLs.
Closes#7761.
## Summary
This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.
Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.
### Index priority
Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.
Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.
You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.
### Index pinning
If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[tool.uv.sources]
torch = { index = "pytorch" }
```
If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch" }
```
Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.
(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)
### Default index
By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```
### Name reuse
If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.
For example, given:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```
The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.
Closes#171.
## Future work
- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
{ index = "cpu", markers = "sys_platform == 'darwin'" },
{ index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
## Summary
This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.
I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.
This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.
I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.
I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.
Did I miss any? Are their initial docs cohesive?
Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.
## Test Plan
Existing tests
This PR adds some additional sanity checking on resolution graphs to
ensure we can never install different versions of the same package into
the same environment.
I used code similar to this to provoke bugs in the resolver before the
release, but it never made it into `main`. Here, we add the error
checking to the creation of `ResolutionGraph`, since this is where it's
most convenient to access the "full" markers of each distribution.
We only report an error when `debug_assertions` are enabled to avoid
rendering `uv` *completely* unusuable if a bug were to occur in a
production binary. For example, maybe a conflict is detected in a marker
environment that isn't actually used. While not ideal, `uv` is still
usable for any other marker environment.
Closes#5598
Simon Sure
Charlie Marsh
konsti
Andrew Gallant
Zanie Blue
Eric Mark Martin
samypr100