Check that the source and module directory exist when build a source
distribution, instead of delaying the check to building the wheel. This
prevents building source distributions that can never be built into
wheels.
I removed the `set_cksum` as the value of it is replaced inside of
`append_data`.
## Summary
This should fix#12762 but I don't know how to test it.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
I think the lack of enforcement here is an oversight. We _do_ already
enforce this for user-level configuration files (contrary to the issue
-- at least, in my testing and from reading the code).
Closes https://github.com/astral-sh/uv/issues/12753.
## Summary
See the test cases. Previously, you could end up with something like:
```toml
[tool.uv.workspace]
members = [
"foo",
"bar",
"baz", "bop",
]
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes#12606.
Two options considered, thanks to @zanieb's guidance are:
1. Special-casing on parse error and encountering the `environment.yml`
filename, possibly at `RequirementsTxt::parse`
2. Adding a new `RequirementsSource::EnvironmentYml` variant and
erroring on `RequirementSpecification::from_source`
I went with the latter for the following reasons:
- This edge case is explicitly modelled within the type system. However,
it changes the semantics of `RequirementsSource` to also model
_unsupported_ sources.
- (**Separation of concerns**) The special-casing would occur in the
`uv-requirements-txt` crate, which seems to be relatively deep in the
guts of the codebase. In my opinion, maintainers working in
`uv-requirements-txt` would reasonably assume the input file to be a
`requirements.txt` file, instead of having to be concerned with it being
another file format (`environment.yml`, `pyproject.toml`, etc.)
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Manually tested as follows:
```sh
>>> cargo run -- pip install -r environment.yml
error: Conda environment file `environment.yml` is not supported
>>> cargo run -- add -r environment.yml
error: Conda environment file `environment.yml` is not supported
```
If you can point me to the appropriate test module, I can write up tests
for these to use `insta`.
<!-- How was it tested? -->
## Summary
Add an option to overwrite the list of available Python downloads from a
local JSON file by using the environment variable
`UV_PYTHON_DOWNLOADS_JSON_URL`
as an experimental support for providing custom sources for Python
distribution binaries #8015
related #10203
I probably should make the JSON to be fetched from a remote URL instead
of a local file.
please let me know what you think and I will modify the code
accordingly.
## Test Plan
### normal run
```
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.14.0a4+freethreaded-linux-x86_64-gnu <download available>
cpython-3.14.0a4-linux-x86_64-gnu <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu <download available>
cpython-3.13.1-linux-x86_64-gnu <download available>
cpython-3.12.8-linux-x86_64-gnu <download available>
cpython-3.11.11-linux-x86_64-gnu <download available>
cpython-3.10.16-linux-x86_64-gnu <download available>
cpython-3.9.21-linux-x86_64-gnu <download available>
cpython-3.8.20-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
pypy-3.10.14-linux-x86_64-gnu <download available>
pypy-3.9.19-linux-x86_64-gnu <download available>
pypy-3.8.16-linux-x86_64-gnu <download available>
pypy-3.7.13-linux-x86_64-gnu <download available>
```
### empty JSON file
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL
{}
root@75c66494ba8b:/# /code/target/release/uv python list
root@75c66494ba8b:/#
```
### JSON file with valid version
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL
{
"cpython-3.11.9-linux-x86_64-gnu": {
"name": "cpython",
"arch": {
"family": "x86_64",
"variant": null
},
"os": "linux",
"libc": "gnu",
"major": 3,
"minor": 11,
"patch": 9,
"prerelease": "",
"url": "https://github.com/astral-sh/python-build-standalone/releases/download/20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
"sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9",
"variant": null
}
}
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.11.9-linux-x86_64-gnu <download available>
root@75c66494ba8b:/#
```
### Remote Path
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json
root@75c66494ba8b:/# /code/target/release/uv python list
error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix)
```
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
## Summary
closes#12234
[fetch_with_cli](e0f81f0d4a/crates/uv-git/src/git.rs (L573))
doesn't respect the registry client's [connectivity
setting](e0f81f0d4a/crates/uv-client/src/registry_client.rs (L1009))
- this pr updates `fetch_with_cli` to set `GIT_ALLOW_PROTOCOL=file` when
the client's connectivity setting is `Connectivity::Offline`
## Test Plan
E2E
```sh
cargo run add "pycurl @ git+https://github.com/pycurl/pycurl.git" --directory ~/src/offline-test/ --offline
```
```sh
Compiling uv-cli v0.0.1 (/Users/justinchapman/src/uv/crates/uv-cli)
Compiling uv v0.6.11 (/Users/justinchapman/src/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.47s
Running `target/debug/uv add 'pycurl @ git+https://github.com/pycurl/pycurl.git' --directory /Users/justinchapman/src/offline-test/ --offline`
Updating https://github.com/pycurl/pycurl.git (HEAD) × Failed to download and build `pycurl @ git+https://github.com/pycurl/pycurl.git`
├─▶ Git operation failed
├─▶ failed to fetch into: /Users/justinchapman/.cache/uv/git-v0/db/9a596e5213c3162d
╰─▶ process didn't exit successfully: `/usr/bin/git fetch --force --update-head-ok 'https://github.com/pycurl/pycurl.git' '+HEAD:refs/remotes/origin/HEAD'` (exit status: 128)
--- stderr
fatal: transport 'https' not allowed
help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
@jtfmumm mentioned a desire for this. I'm not sure how we should do
this. I kind of want to change this to something like...
```
$ uv python find
CPython 3.13 @ <path>
$ uv python find --only-path
<path>
$ uv python find --short
<path>
$ uv python find --only-version
3.13
```
The change in defaults would be breaking though.
uv has a global `--no-config` option, which indeed has an effect.
## Test Plan
```console
❯ cat pyproject.toml
[project]
name = "hello"
version = "0.1.0"
[[tool.uv.index]]
url = "http://non-exist-host.com/simple"
default = true
❯ echo requests | uv pip compile -
⠦ Resolving dependencies... error: Failed to fetch: `http://non-exist-host.com/simple/requests/`
Caused by: Could not connect, are you offline?
Caused by: Request failed after 3 retries
Caused by: error sending request for url (http://non-exist-host.com/simple/requests/)
Caused by: client error (Connect)
Caused by: dns error: failed to lookup address information: nodename nor servname provided, or not known
Caused by: failed to lookup address information: nodename nor servname provided, or not known
# Before
❯ echo requests | uv pip compile --no-config -
warning: pip-compile's `--no-config` has no effect (uv does not use a configuration file)
Resolved 5 packages in 13ms
# This file was autogenerated by uv via the following command:
# uv pip compile --no-config -
certifi==2025.1.31
# via requests
charset-normalizer==3.4.1
# via requests
idna==3.10
# via requests
requests==2.32.3
urllib3==2.3.0
# via requests
# After
❯ echo requests | uv pip compile --no-config -
Resolved 5 packages in 13ms
# This file was autogenerated by uv via the following command:
# uv pip compile --no-config -
certifi==2025.1.31
# via requests
charset-normalizer==3.4.1
# via requests
idna==3.10
# via requests
requests==2.32.3
urllib3==2.3.0
# via requests
```
## Summary
Fix a suggestion in the docs on configs through environment variables,
which lists an option value that doesn't appear to exist.
The description implies that `unsafe-best-match` was intended here.
## Test Plan
Verified by providing `unsafe-any-match` as a parameter to `uv`. It
didn't error, but appeared to use the `first-index` strategy instead.
The value I changed it to behaves as described by the documentation.
In an attempt to avoid reporting shims as their resolved
`sys.executable` path, we report the queried executable path instead.
This seems more correct for this command, broadly? Interestingly, it
changes the reported paths for Homebrew Python
<img width="1430" alt="Screenshot 2025-04-02 at 11 05 18 AM"
src="https://github.com/user-attachments/assets/0e1600e8-fb07-40c7-a6d6-56eaeb4b9293"
/>
Closes#9979
This will in principle fix the problem reported in #12611 that
`authenticate = "always"` is ignored for an index when `explicit =
true`. This change ensures all indexes are added to the URL auth
policies list passed to our auth middleware.
Incorporates #12624Fixes#12611
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Fixes#12618
Instead of succeeding the user now gets:
```
uvdloc pip install osqp==1.0.2 --reinstall --python-platform=linux
Resolved 7 packages in 171ms
× Failed to download `osqp==1.0.2`
├─▶ Failed to extract archive
╰─▶ a computed CRC32 value did not match the expected value
```
I am not entirely sure if we have infra for testing this kind of thing,
but it would be nice to check in a test or two. I'm also not entirely
clear if there's any cases where these checks are overzealous.
## Summary
I noticed that these only support Basic credentials, but we may want to
allow users to provide Bearer tokens? This PR just generalizes the type.
serde needs to be told where to put underscores. someone clearly noticed
this when adding attributes for schemars, but they need to be present
for serde too and then schemars gets them for free.
Strictly speaking this would be a breaking change for anyone who noticed
the parsing was messed up and worked around it. So we add aliases for
backcompat, at least for a few releases.
Fixes#12590
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Support the `UV_PROJECT` environment variable to set the project
directory.
#11946
## Test Plan
<!-- How was it tested? -->
`cargo nextest run` passed except the cache_prune.
```
export UV_PROJECT=/path/to/project
uv sync
```
works.
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
The overall strategy here is to make this code look more like
`requirements_txt.rs`: we seed the root members, then perform a DFS.
Previously, we created all nodes upfront, which caused problems when
using `--only-group`, since we'd omit "production" dependencies of
development dependencies.
Closes https://github.com/astral-sh/uv/issues/12526.
Adding dependency trace/parent comments ("via ...") to the export
command output.
This is a similar behavior to the pip compile output.
#### Note to the eager reviewer:
First of all - thanks!
Secondly, this is still a very rough draft. These are the first lines of
code I've ever written in Rust. This is still mostly an educational/fun
exercise for myself. If opening a Draft PR is creating too much noise -
I apologize and I will close it until it is ready.
## Summary
Resolves#7777
## Test Plan
- [X] manual command execution
- [x] update expected output in tests
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
uv doesn't separate the metadata block from other blocks when adding the
`script` block to a script, which results in the next block being
considered part of the script block and causes errors when running.
See #12499 for more details.
Closes#12499
## Test Plan
I manually tested the most common scenario, but there's a few edge cases
that would be good to have tests for.
I would have written the tests also, but I was running into errors like
this:
```bash
$ cargo test --package uv-scripts
Compiling uv-configuration v0.0.1 (/home/merlin/Projects/uv/crates/uv-configuration)
error: cannot find attribute `value` in this scope
--> crates/uv-configuration/src/project_build_backend.rs:8:38
|
8 | #[cfg_attr(feature = "schemars", value(hide = true))]
| ^^^^^
error: could not compile `uv-configuration` (lib) due to 1 previous error
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
We tend not to run tests for individual crates, which can lead to weird
situations like this, where crates are missing optional features that
are otherwise installed globally.
## Test Plan
Run `cargo test --profile fast-build -p uv-scripts`, which otherwise
fails to compile.
The idea here is that we introduce a new stdout_important method for
things that want to care about the difference between "quiet" and
"silent".
This PR is WIP because it has no actual uses of stdout_important, and we
should have at least one before landing this. Perhaps someone has a
suggestion for commands that would really benefit from this distinction?
Fixes#10431
## Summary
This PR extends `[[tool.uv.index]]` to support `--find-links`-style
"flat" indexes, so that users can point to such indexes without using
`--find-links` _and_ get access to the full functionality of
`[[tool.uv.index]]` (e.g., they can now pin packages to
`--find-links`-style indexes).
Note that, at present, `--find-links` indexes actually have some quirky
behavior, in that we combine them into a single entity and then merge
the discovered distributions into each Simple API-style index. The
motivation here, IIRC, was to match pip's behavior quite closely. I'm
interested in _removing_ that behavior, but it'd be breaking (and may
also be inconvenient for some use-cases). So, the behavior for indexes
passed in via `--find-links` remains completely unchanged. However,
`[[tool.uv.index]]` entries with `format = "flat"` are now treated
identically to those defined with `format = "simple"` (the default), in
that we stop after we find the first-matching index, etc.
Closes https://github.com/astral-sh/uv/issues/11634.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
We were not applying the `authenticate = "always"` behavior to `uv pip`
commands (related to #12362). This PR addresses that, applying
authentication policies wherever we set up a registry client.
## Summary
This fixes https://github.com/astral-sh/uv/issues/12426 which helps use
a more accurate arg name in the help output.
## Test Plan
I didn't test it locally, @charliermarsh gave me guidance on what to
change so I looked around that file for another example of `value_name`
and repeated what I saw. I kept it formatted to 1 line based on it not
being a long line. The other example of `value_name` had everything on
separate lines because there were a bunch of parameters passed in.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This PR modifies the requirement source entities to store a (new)
container struct that wraps `IndexUrl`. This will allow us to store
user-defined metadata alongside `IndexUrl`, and propagate that metadata
throughout resolution.
Specifically, I need to store the "kind" of the index (Simple API vs.
`--find-links`), but I also ran into this problem when I tried to add
support for overriding `Cache-Control` headers on a per-index basis: at
present, we have no way to passing around metadata alongside an
`IndexUrl`.
Match the module name to its module directory with potentially different
casing.
For example, a package may have the dist-info-normalized package name
`pil_util`, but the importable module is named `PIL_util`.
We get the module name either as dist-info-normalized package name, or
explicitly from the user. For dist-info-normalizing a package name, the
rules are lowercasing, replacing `.` with `_` and replace `-` with `_`.
Since `.` and `-` are not allowed in module names, we can check whether
a directory name matches our expected module name by lowercasing it.
Fixes#12187
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
In general, we merge `--find-links` entries into each index. If a
package is pinned to an index, though, it seems surprising (and wrong)
that we'd ever select a distribution from `--find-links`. This PR
modifies the provider to ignore `--find-links` for any explicitly pinned
packages.
Allows `uv python list <request>` to filter the installed list. I often
want this and it's not hard to add.
I tested the remote download filtering locally (#12381 is needed for
snapshot tests)
```
❯ cargo run -q -- python list --all-versions 3.13
cpython-3.13.2-macos-aarch64-none <download available>
cpython-3.13.1-macos-aarch64-none /opt/homebrew/opt/python@3.13/bin/python3.13 -> ../Frameworks/Python.framework/Versions/3.13/bin/python3.13
cpython-3.13.1-macos-aarch64-none <download available>
cpython-3.13.0-macos-aarch64-none /Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/bin/python3.13
❯ cargo run -q -- python list --all-versions 3.13 --only-installed
cpython-3.13.1-macos-aarch64-none /opt/homebrew/opt/python@3.13/bin/python3.13 -> ../Frameworks/Python.framework/Versions/3.13/bin/python3.13
cpython-3.13.0-macos-aarch64-none /Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/bin/python3.13
```
## Summary
I want to use the flat index client from within the registry client, so
making them both depend on the same underlying primitives rather than
having the flat index client depend on the registry client.
## Summary
We respect `--exclude-newer` during resolution, but we weren't applying
it to individual _files_ when writing the lockfile. As a result, if
wheels were added to a distribution after its initial release, we'd end
up including them in the lockfile, even if they were uploaded after the
`--exclude-newer` date.
Closes https://github.com/astral-sh/uv/issues/12296.
## Summary
It's possible that the PyTorch version the user depends on isn't in the
latest index. These indexes are equally trusted, so we should override
the policy.
Closes#12357.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This fixes a case described in #12333, where trailing comments in
dependencies can be unexpectedly shifted when a new dependency is added.
Fixes#12333.
## Test Plan
<!-- How was it tested? -->
`cargo test` (Added a snapshot test)
## Summary
Resolves#11794.
When `uv python find` is given a `--script` option, either the existing
environment for that script or the Python executable that would be used
to create it will be returned. If neither are found, the command exits
with exit code 1.
`--script` is incompatible with all other options to the same command.
## Test Plan
Unit tests.
## Summary
This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
There was a bug where `UV_MANAGED_PYTHON` and `UV_NO_MANAGED_PYTHON`
only accepted `true` or `false`. This switches to the boolish value
parser for those flags.
Closes#12336
## Summary
Fixes the failing `cache_prune::prune_unzipped` test that was causing CI
failures in my other PR (#12328) and others like PR #12327.
The error message format changed to show a specific version constraint
(`iniconfig<=2.0.0`) rather than the generic 'all versions' message.
This PR updates the test to expect the new, more specific error message.
## Test Plan
Ran `cargo test -p uv cache_prune::prune_unzipped` to verify the test
now passes.
Previously, we required a username to perform a fetch from the keyring
because the `keyring` CLI only supported fetching password for a given
service and username. Unfortunately, this is different from the keyring
Python API which supported fetching a username _and_ password for a
given service. We can't (easily) use the Python API because we don't
expect `keyring` to be installed in a specific environment during
network requests. This means that we did not have parity with `pip`.
Way back in https://github.com/jaraco/keyring/pull/678 we got a `--mode
creds` flag added to `keyring`'s CLI which supports parity with the
Python API. Since `keyring` is expensive to invoke and we cannot be
certain that users are on the latest version of keyring, we've not added
support for invoking keyring with this flag. However, now that we have a
mode that says authentication is _required_ for an index (#11896), we
might as well _try_ to invoke keyring with `--mode creds` when there is
no username. This will address use-cases where the username is
non-constant and move us closer to `pip` parity.
This addresses a small part of #12280, namely when you have
`authenticate` set to `always`, it will output a distinct error message
for the case where you have a username but are missing a password.
## Summary
This is a prototype that I'm considering shipping under `--preview`,
based on [`light-the-torch`](https://github.com/pmeier/light-the-torch).
`light-the-torch` patches pip to pull PyTorch packages from the PyTorch
indexes automatically. And, in particular, `light-the-torch` will query
the installed CUDA drivers to determine which indexes are compatible
with your system.
This PR implements equivalent behavior under `--torch-backend auto`,
though you can also set `--torch-backend cpu`, etc. for convenience.
When enabled, the registry client will fetch from the appropriate
PyTorch index when it sees a package from the PyTorch ecosystem (and
ignore any other configured indexes, _unless_ the package is explicitly
pinned to a different index).
Right now, this is only implemented in the `uv pip` CLI, since it
doesn't quite fit into the lockfile APIs given that it relies on feature
detection on the currently-running machine.
## Test Plan
On macOS, you can test this with (e.g.):
```shell
UV_TORCH_BACKEND=auto UV_CUDA_DRIVER_VERSION=450.80.2 cargo run \
pip install torch --python-platform linux --python-version 3.12
```
On a GPU-enabled EC2 machine:
```shell
ubuntu@ip-172-31-47-149:~/uv$ UV_TORCH_BACKEND=auto cargo run pip install torch -v
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.31s
Running `target/debug/uv pip install torch -v`
DEBUG uv 0.6.6 (e95ca063b 2025-03-14)
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `cpython-3.13.0-linux-x86_64-gnu` at `/home/ubuntu/uv/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.13.0 environment at: .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: torch
warning: The `--torch-backend` setting is experimental and may change without warning. Pass `--preview` to disable this warning.
DEBUG Detected CUDA driver version from `/sys/module/nvidia/version`: 550.144.3
...
```
## Summary
I think this is reasonable to change. Right now, if you're on Python
3.11, the resolver returns `multiprocess-0.70.17-py311-none-any.whl`,
but `multiprocess-0.70.17-py310-none-any.whl` is in the cache, we'll
reuse `multiprocess-0.70.17-py310-none-any.whl` (since it _is_
compatible with Python 3.11).
Instead, we now _require_ the cached wheel to match the wheel returned
by the resolver.
Closes https://github.com/astral-sh/uv/issues/12273.
Currently, for users to specify at the command line whether to use
uv-managed or system Python interpreters, they use the
`--python-preference` parameter, which takes four possible values. This
is more complex than necessary since the normal case is to either say
"only managed" or "not managed". This PR hides the old
`--python-preference` parameter from help and documentation and adds two
new flags: `--managed-python` and `--no-managed-python` to capture the
"only managed" and "not managed" cases.
I have successfully tested this locally but currently cannot add
snapshot tests because of problems with distinguishing managed vs.
system interpreters in CI (and non-determinism when run on different
developers' machines). The `--python-preference` test in
`tool-install.rs` is currently ignored for this reason. See #5144 and
#7473.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
I suspect this only affects packages with quotes in the requires-python,
which is typically an error but one that we correct for in
`LenientVersionSpecifiers`.
Closes https://github.com/astral-sh/uv/issues/12260.
Reject lockfiles where the package version and the wheel versions are
incoherent. This implicitly checks that all wheel files have the same
version.
It does not check for the source dist version, since a source dist may
not contain a version in the filename and attempting to deserialize
source dist filenames we may not need is a performance overhead for
something that's already slow in `uv run`.
Fixes#12164
## Summary
This has come up a few times, so it seems worth addressing. If you
migrate from a flat layout to a `src` layout or vice versa, we now
invalidate the package metadata.
Closes https://github.com/astral-sh/uv/issues/12047
## Summary
This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)
Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.
Closes#12038.
This is a minimal redux of #10861 to be compatible with `uv pip`.
This implements the interface described in:
https://github.com/pypa/pip/pull/13065#issuecomment-2544000876 for `uv
pip install` and `uv pip compile`. Namely `--group <[path:]name>`, where
`path` when not defined defaults to `pyproject.toml`.
In that interface they add `--group` to `pip install`, `pip download`,
and `pip wheel`. Notably we do not define `uv pip download` and `uv pip
wheel`, so for parity we only need to implement `uv pip install`.
However, we also support `uv pip compile` which is not part of pip
itself, and `--group` makes sense there too.
----
The behaviour of `--group` for `uv pip` commands makes sense for the
cases upstream pip supports, but has confusing meanings in cases that
only we support (because reading pyproject.tomls is New Tech to them but
heavily supported by us). **Specifically case (h) below is a concerning
footgun, and case (e) below may get complaints from people who aren't
well-versed in dependency-groups-as-they-pertain-to-wheels.**
## Only Group Flags
Group flags on their own work reasonably and uncontroversially, except
perhaps that they don't do very clever automatic project discovery.
a) `uv pip install --group path/to/pyproject.toml:mygroup` pulls up
`path/to/project.toml` and installs all the packages listed by its
`mygroup` dependency-group (essentially treating it like another kind of
requirements.txt). In this regard it functions similarly to
`--only-group` in the rest of uv's interface.
b) `uv pip install --group mygroup` is just sugar for `uv pip install
--group pyproject.toml:mygroup` (**note that no project discovery
occurs**, upstream pip simply hardcodes the path "pyproject.toml" here
and we reproduce that.)
c) `uv pip install --group a/pyproject.toml:groupx --group
b/pyproject.toml:groupy`, and any other instance of multiple `--group`
flags, can be understood as completely independent requests for the
given groups at the given files.
## Groups With Named Packages
Groups being mixed with named packages also work in a fairly
unsurprising way, especially if you understand that things like
dependency-groups are not really supposed to exist on pypi, they're just
for local development.
d) `uv pip install mypackage --group path/to/pyproject.toml:mygroup`
much like multiple instances of `--group` the two requests here are
essentially completely independent: pleases install `mypackage`, and
please also install `path/to/pyproject.toml:mygroup`.
e) `uv pip install mypackage --group mygroup` is exactly the same, but
this is where it becomes possible for someone to be a little confused,
as you might think `mygroup` is supposed to refer to `mypackage` in some
way (it can't). But no, it's sourcing `pyproject.toml:mygroup` from the
current working directory.
## Groups With Requirements/Sourcetrees/Editables
Requirements and sourcetrees are where I expect users to get confused.
It behaves *exactly* the same as it does in the previous sections but
you would absolutely be forgiven for expecting a different behaviour.
*Especially* because `--group` with the rest of uv *does* do something
different.
f) `uv pip install -r a/pyproject.toml --group b/pyproject.toml:mygroup`
is again just two independent requests (install `a/pyproject.toml`'s
dependencies, and `b/pyproject.toml`'s `mygroup`).
g) `uv pip install -r pyproject.toml --group mygroup` is exactly like
the previous case but *incidentally* the two requests refer to the same
file. What the user wanted to happen is almost certainly happening, but
they are likely getting "lucky" here that they're requesting something
simple.
h) `uv pip install -r a/pyproject.toml --group mygroup` is again exactly
the same but the user is likely to get surprised and upset as this
invocation actually sources two different files (install
`a/pyproject.toml`'s dependencies, and `pyproject.toml`'s `mygroup`)! I
would expect most people to assume the `--group` flag here is covering
all applicable requirements/sourcetrees/editables, but no, it continues
to be a totally independent reference to a file with a hardcoded
relative path.
------
Fixes https://github.com/astral-sh/uv/issues/8590
Fixes https://github.com/astral-sh/uv/issues/8969
## Summary
Closes https://github.com/astral-sh/uv/issues/12178
## Test Plan
Added new test. Manually tested on Windows and Linux.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This PR enables module name overrides for editable installs.
Builds upon https://github.com/astral-sh/uv/pull/11884. The
`tool.uv.build-backend.module-name` option is now respected during
editable build processes.
## Test Plan
Added a test.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
There were no GraalPy binary wheels were available when uv support was
added, and thus the abi tag was never tested against actual packages.
Now that GraalPy publishes binary wheels to
https://www.graalvm.org/python/wheels/ we noticed the abi tag was
incorrect and the version info incorrectly determined.
## Test Plan
I tested manually:
```
> target/debug/uv venv --python graalpy testvenv
Using GraalPy 3.11.7 interpreter at: /home/tim/.pyenv/versions/graalpy-24.1.1/bin/graalpy
Creating virtual environment at: testvenv
Activate with: source testvenv/bin/activate
> cat <<EOF> uv.toml
> [[index]]
> url = "https://www.graalvm.org/python/wheels/"
> EOF
> target/debug/uv -v pip install psutil
warning: Found both a `uv.toml` file and a `[tool.uv]` section in an adjacent `pyproject.toml`. The `[tool.uv]` section will be ignored in favor of the `uv.toml` file.
DEBUG uv 0.6.6+3 (be8725553 2025-03-13)
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `graalpy-3.11.7-linux-x86_64-gnu` at `/home/tim/dev/uv/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.11.7 environment at: .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: psutil
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.7
DEBUG Solving with target Python version: >=3.11.7
DEBUG Adding direct dependency: psutil*
DEBUG Found fresh response for: https://www.graalvm.org/python/wheels/psutil/
DEBUG Searching for a compatible version of psutil (*)
DEBUG Selecting: psutil==5.9.8 [compatible] (psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl)
DEBUG No cache entry for: https://gds.oracle.com/download/graalpy-wheels/psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl
DEBUG Tried 1 versions: psutil 1
DEBUG marker environment resolution took 0.968s
Resolved 1 package in 971ms
DEBUG Identified uncached distribution: psutil==5.9.8
DEBUG No cache entry for: https://gds.oracle.com/download/graalpy-wheels/psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl
Prepared 1 package in 268ms
Installed 1 package in 28ms
+ psutil==5.9.8
DEBUG Released lock at `/home/tim/dev/uv/.venv/.lock`
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
In #10669, a pyproject.toml with requires-python but no environment had
a lockfile covering only a subset of the requires-python space:
```toml
resolution-markers = [
"python_full_version >= '3.10' and platform_python_implementation == 'CPython'",
"python_full_version == '3.9.*'",
"python_full_version < '3.9'",
]
```
This marker set is invalid, we have to reject the lockfile. (We can
still use the versions though, to avoid churn).
Part 1/2 of #10669
These changes add support for
```
uv python pin 3.12 --global
```
This adds the specified version to a `.python-version` file in the
user-level config directory. uv will now use the user-level version as a
fallback if no version is found in the project directory or its
ancestors.
Closes#4972
When making changes to uv that require new (or altered) settings, there
are many places in the code that need to change. This slows down work,
reduces confidence in changes for new developers, and adds noise to PRs.
The goal of this PR is to reduce the number of points that need to
change (and that the developer needs to understand) when making changes
to settings.
This PR consolidates `ResolverSettings` and `ResolverInstallerSettings`
by factoring out the shared settings and using a new field
`resolver_settings` on `ResolverInstallerSettings`. This not only
reduces repetition, but makes it easier for a human to parse the code
without having to compare long lists of fields to spot differences (the
difference was that `ResolverInstallerSettings` had two extra fields).
This also removes `ResolverSettingsRef` and
`ResolverInstallerSettingsRef`, using normal Rust references instead.
For the time being, I've left `InstallerSettingsRef` in place because it
appears to have a semantic meaning that might be relied upon. However,
it would now be straightforward to refactor to pass
`&ResolverInstallerSettings` wherever `InstallerSettingsRef` appears,
further reducing sprawl.
The change has the downside of adding
`settings.resolver_settings.<field>` and requiring dereferencing at
various points where it was not required before (with the *SettingsRef
approach). But this means there are significantly fewer places that must
change to update settings.
Following the upstream release and #12120, removes gating preventing
installation of the managed musl Python versions.
Of note
- The filtering of musl Python distributions has moved from the Rust
runtime to the metadata fetcher
- The filtering is now conditional on the PBS release date, removing all
old static musl distributions
- We could support the `+static` musl downloads in the future; right
now, they are deprioritized when selecting a variant
- I added test to CI which uses Alpine and installs numpy
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR is meant to fix issue #11862
It allows to send sized bodies during `publish`
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
the PR was tested on the MRE from #11862
<!-- How was it tested? -->
## Summary
Follow up to https://github.com/astral-sh/uv/pull/11888 with added
support for uv tool run.
Changes
* Added functionality for running windows scripts in previous PR was
moved from run.rs to uv_shell::runnable.
* EXE was added as a supported type, this simplified integration across
both uv run and uvx while retaining a backwards compatible behavior and
properly prioritizing .exe over others. Name was adjusted to runnable as
a result to better represent intent.
## Test Plan
New tests added.
## Documentation
Added new documentation.
Reduce the overhead of `uv run` in large workspaces. Instead of
re-discovering the entire workspace each time we resolve the metadata of
a member, we can the discovered set of workspace members. Care needs to
be taken to not cache the discovery for `uv init`, `uv add` and `uv
remove`, which change the definitions of workspace members.
Below is apache airflow e3fe06382df4b19f2c0de40ce7c0bdc726754c74 `uv run
python` with a minimal payload. With this change, we avoid a ~350ms
overhead of each `uv run` invocation.
```
$ hyperfine --warmup 2 \
"uv run --no-dev python -c \"print('hi')\"" \
"uv-profiling run --no-dev python -c \"print('hi')\""
Benchmark 1: uv run --no-dev python -c "print('hi')"
Time (mean ± σ): 492.6 ms ± 7.0 ms [User: 393.2 ms, System: 97.1 ms]
Range (min … max): 482.3 ms … 501.5 ms 10 runs
Benchmark 2: uv-profiling run --no-dev python -c "print('hi')"
Time (mean ± σ): 129.7 ms ± 2.5 ms [User: 105.4 ms, System: 23.2 ms]
Range (min … max): 126.0 ms … 136.1 ms 22 runs
Summary
uv-profiling run --no-dev python -c "print('hi')" ran
3.80 ± 0.09 times faster than uv run --no-dev python -c "print('hi')"
```
The profile after those change below. We still spend a large chunk in
toml parsing (both `uv.lock` and `pyproject.toml`), but it's not
excessive anymore.
Adds a new optional key `auth-policy` to `[tool.uv.index]` that sets the
authentication policy for the index URL.
The default is `"auto"`, which attempts to authenticate when necessary.
`"always"` always attempts to authenticate and fails if the endpoint is
unauthenticated. `"never"` never attempts to authenticate.
These policy address two kinds of cases:
* Some indexes don’t fail on unauthenticated requests; instead they just
forward to the public PyPI. This can leave the user confused as to why
their package is missing. The "always" policy prevents this.
* "never" allows users to ensure their credentials couldn't be leaked to
an unexpected index, though it will only allow for successful requests
on an index that doesn't require credentials.
Closes#11600
In the publish client, we have to set the client retries to 0 as the
retry middleware is incompatible with upload bodies. This however also
sets `client.retry_policy()` to a zero-retry policy, so we need to
construct our own policy.
Fixes#12027
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This PR moves functions for finding user- and system-level config
directories to public functions in `uv_fs::config`. This will allow them
to be used in future work without duplicating code.
This adds support for inferring dependency group conflict sets from the
directly defined conflicts in configuration. For example, if you declare
a conflict between groups `alpha` and `beta` and `dev` includes `beta`,
then we will infer a conflict between `dev` and `alpha`. We will also
handle a conflict between two groups if they transitively include groups
that conflict with each other. See #11232 for more details.
Closes#11232
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
The way the `tool update-shell` checks if the command to export the PATH
exists or not in the RC files is a blind search, and therefore if finds
the command inside comments.
example with .zshenv
This content
```
# uv
# export PATH="/Users/cholas/.local/bin:$PATH"
```
Generates the following msg
```
error: The executable directory /Users/cholas/.local/bin is not in PATH, but the Zsh configuration files are already up-to-date
```
With this change, that content won't be considered as configured and the
following will be added
```
# uv
export PATH="/Users/cholas/.local/bin:$PATH"
```
This will make the `update-shell` more reliable
## Test Plan
I tested with and without the change with commented export in zsh in
mac. Tested running `cargo run -- tool update-shell`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
I somehow missed running an actual integration test of the PEP 517 API
in CI and the python shim was using the old uv CLI interface still.
The tests include pip, uv and `python -m build`. They must be a in CI
job since we can't depend on the Python package in the Rust tests (we
only get the binary in `cargo test`, not the `uv_build` wheel).
## Summary
The order here is slightly off... As-is, we fetch the metadata for the
dependency, _then_ insert the URLs and indexes into the fork state -- so
the fetch doesn't take the explicit index or URL into account. This has
mostly been unobserved because we re-fetch anyway in the next request,
but if we do things in the right order (add to fork state, fetch
dependencies, insert dependencies), we can cut down on the fetches.
Closes https://github.com/astral-sh/uv/issues/12056.
Thank you for uv, it has game-changer capabilities in the field of
Python package and environment maangement!
## Summary
This is a small PR adding the option `module-name`
(`tool.uv.build-backend.module-name`) to the uv build backend (
https://github.com/astral-sh/uv/issues/8779 ).
Currently, the uv build backend will assume that the module name matches
the (dash to underdash-transformed) package name. In some packaging
scenarios this is not the case, and currently there exists no
possibility to override it, which this PR addresses.
From the main issue ( https://github.com/astral-sh/uv/issues/8779 ) I
could not tell if there is any extensive roadmap or plans how to
implement more complex scenarios, hence this PR as a suggestion for a
small feature with a big impact for certain scenarios.
I am new to Rust, I hope the borrow/reference usage is correct.
## Test Plan
So far I tested this at an example, if desired I can look into extending
the tests.
Fixes#11428
---------
Co-authored-by: konstin <konstin@mailbox.org>
This PR is in support of #12005, where we need to import
`DependencyGroups` in the `uv-pypi-types` crate without a circular
dependency on `uv-workspace`.
uv itself is a large package with many dependencies and lots of
features. To build a package using the uv build backend, you shouldn't
have to download and install the entirety of uv. For platform where we
don't provide wheels, it should be possible and fast to compile the uv
build backend. To that end, we're introducing a python package that
contains a trimmed down version of uv that only contains the build
backend, with a minimal dependency tree in rust.
The `uv_build` package is publish from CI just like uv itself. It is
part of the workspace, but has much less dependencies for its own
binary. We're using cargo deny to enforce that the network stack is not
part of the dependencies. A new build profile ensure we're getting the
minimum possible binary size for a rust binary.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
In https://github.com/astral-sh/uv/issues/11998, a user is attempting to
vendor `pydantic-core`. But when they add `pydantic-core = { path =
"src/foo/vendor/pydantic-core" } `, we're installing it as a virtual
package, since `pydantic-core/pyproject.toml` contains `package =
false`.
This PR allows users to mark dependencies as "explicitly a package" or
"explicitly not a package" (i.e., virtual), as a workaround.
Closes https://github.com/astral-sh/uv/issues/11998.
## Summary
This PR adds support for `SitePackages::satisfies` with unnamed
overrides and requirements.
The main challenge here was cases like: you have a `requirements.in`
with `git+https://github.com/pallets/flask` in it, and an
`overrides.txt` with `flask==2.0.0` in it. You _need_ to include
`flask==2.0.0`, but you can't know that without resolving the unnamed
URL requirement (since overrides only take effect when the package is
included, like constraints).
We now make the assumption that any unnamed overrides _are_ relevant,
for the purpose of the satisfies check. This is conservative, but this
whole check is an optimization anyway.
## Summary
Closes https://github.com/astral-sh/uv/issues/9151
This adds support for running .ps1, .cmd, .bat legacy scripts typically
provided by setuptools [legacy script
files](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#scripts).
Note, .bat and .cmd scripts were somewhat supported previously by
[Command](https://doc.rust-lang.org/std/process/index.html#batch-file-special-handling)
when the extension was explicit but documentation says such behavior
should not be relied upon.
In addition, when no extension is provided and a legacy script exists,
it will try to infer the appropriate extension on Windows and use the
right runtime with preference for .ps1. Only powershell.exe and cmd.exe
are supported right now.
## Test Plan
Added tests. Tested with nuitka locally via uv run.
Note uvx support will be added in a follow up.
## Summary
Small omission I noticed last night. This was overly strict (so, didn't
lead to any incorrect behavior; more that we did unnecessary work in
some cases).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Similar to https://github.com/astral-sh/uv/pull/11399
This adds `UV_NO_BUILD` and `UV_NO_BUILD_PACKAGE` environment variables
for non-pip commands.
## Test Plan
<!-- How was it tested? -->
Tested manually and with snapshot tests.
Fixes#11963
Signed-off-by: Alex Lowe <alex@lowe.dev>
## Summary
If we're looking at (e.g.) `python3.12`, and we have a `requires-python:
">=3.12.7, <3.13"`, then checking if the range includes `3.12` will
return `false`. Instead, we need to look at the lower- and upper-bound
major-minors of the `requires-python`.
Closes https://github.com/astral-sh/uv/issues/11825.
We prepend the interpreter discovery in a temporary path to `sys.path`,
which we have to strip to avoid the `sys.path` value containing a
then-deleted temp dir.
If we see `uvx script.py`, we exit early, giving a hint to use `uv run
script.py` if the script exists. If it does not exist, we suggest
running `uv run` with a normalized package name.
This PR includes a snapshot test for each of these scenarios.
An alternative approach would be to wait until we encounter an error,
and then add the hint. But if there happens to be a malicious package
called `script-py`, this would be run unintentionally (a point raised by
@zanieb).
Closes#10784
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fix error message suggesting `--user` instead of `--username`:
```
> uv publish --publish-url ... ... --password $(cat ~/.token)
Publishing 1 file to ...
error: Attempted to publish with a password, but no username. Either provide a username with `--user` (`UV_PUBLISH_USERNAME`), or use `--token` (`UV_PUBLISH_TOKEN`) instead of a password.
> uv publish --publish-url ... ... --user lewis --password $(cat ~/.token)
error: unexpected argument '--user' found
tip: a similar argument exists: '--username'
Usage: uv publish <FILES|--index <INDEX>|--username <USERNAME>|--password <PASSWORD>|--token <TOKEN>|--trusted-publishing <TRUSTED_PUBLISHING>|--keyring-provider <KEYRING_PROVIDER>|--publish-url <PUBLISH_URL>|--check-url <CHECK_URL>|--skip-existing>
For more information, try '--help'.
```
## Test Plan
I have not tested manually, I'm hoping this isn't necessary and there
will be sufficient CI coverage.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
## Summary
I believe `zlib-rs` is now a better choice on ARM and x86, so I'm just
going to assume it's a better choice everywhere. It's much easier to
build (removes our CMake dependency), and in my benchmarking, it's
substantially faster on ARM and faster or ~exactly even on my x86
Windows machine.
We migrated to `zlib-rs` once before (#9184); however, I later reverted
it as I learned that they were only doing compile-time feature
detection, and so `zlib-rs` was meaningfully slower on x86. They now
perform runtime feature detection:
https://trifectatech.org/blog/zlib-rs-is-faster-than-c/.
To benchmark, I wrote a script to create a local Simple API-compliant
registry (see the commit history) for a single package. Then I ran the
`install-cold` benchmark against that registry to install NumPy.
On ARM:
```
❯ uv run resolver --uv-pip-path ../../zlib-ng --uv-pip-path ../../zlib-rs \
--benchmark install-cold \
req.txt --warmup 10 --min-runs 30
Benchmark 1: ../../zlib-ng (install-cold)
Time (mean ± σ): 165.7 ms ± 34.7 ms [User: 64.4 ms, System: 93.2 ms]
Range (min … max): 141.8 ms … 293.2 ms 30 runs
Benchmark 2: ../../zlib-rs (install-cold)
Time (mean ± σ): 150.9 ms ± 16.2 ms [User: 57.4 ms, System: 86.4 ms]
Range (min … max): 135.3 ms … 202.4 ms 30 runs
Summary
../../zlib-rs (install-cold) ran
1.10 ± 0.26 times faster than ../../zlib-ng (install-cold)
```
I benchmarked this about 100 times on my Windows machine and found it
difficult to conclude anything beyond "They're nearly the same". Here's
an example:
```
PS C:\Users\crmar\workspace\puffin> hyperfine --prepare "uv venv" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" --runs 10 --warmup 5
Benchmark 1: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 240.6 ms ± 10.8 ms [User: 6.1 ms, System: 92.2 ms]
Range (min … max): 229.4 ms … 267.9 ms 10 runs
Benchmark 2: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 241.3 ms ± 6.2 ms [User: 7.7 ms, System: 90.6 ms]
Range (min … max): 233.9 ms … 252.1 ms 10 runs
Benchmark 3: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 242.8 ms ± 7.7 ms [User: 6.2 ms, System: 23.4 ms]
Range (min … max): 236.1 ms … 262.8 ms 10 runs
Benchmark 4: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
Time (mean ± σ): 245.9 ms ± 5.7 ms [User: 1.5 ms, System: 59.4 ms]
Range (min … max): 240.9 ms … 257.3 ms 10 runs
Summary
zlib-rs.exe pip sync ./scripts/benchmark/req.txt ran
1.00 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
1.01 ± 0.06 times faster than zlib-rs.exe pip sync ./scripts/benchmark/req.txt
1.02 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
```
Closes#11885.
At certain points in the code, dependency groups are represented by
`DevGroups*` naming, probably as a historical artifact. This PR updates
the naming.
This includes renaming `uv-configuration/src/dev.rs` to
`uv-configuration/src/dependency_groups.rs`.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Follow on to #11706. In the original PR, I tried to solve the issue by
getting rid of the `ctrlc::set_handler` call. Unfortunately, this didn't
work on windows due to an issue with the console crate. console 0.15.11
includes https://github.com/console-rs/console/pull/235, which resolves
the issue, so now we can get rid of the call.
<!-- What's the purpose of the change? What does it do, and why? -->
This change is not super important but I still think it's worthwhile.
For one, spinning up a background thread to handle `SIGINT`s when we're
going to be raising the `SIGINT` from within the function is more
technical complexity than needed, now that there's an easy way to
explicitly catch the Ctrl-C from the terminal input. Secondly,
`ctrlc::set_handler`'s
[docs](https://docs.rs/ctrlc/3.4.5/ctrlc/fn.set_handler.html) advise
that you set the handler just once, at the beginning of the program, so
this use seems somewhat error prone. In fact, uv already has a second
[callsite](461f4d9007/crates/uv/src/commands/project/add.rs (L596-L611))
for this function (though I'm not sure if the two callsites could
currently ever both occur on the same run of uv)
## Test Plan
I've tested this manually on linux (WSL ubuntu) and windows, though not
on aarch64-apple-darwin as I don't have a machine running that. I would
appreciate if someone would double-check that it works on such machines.
As discussed in the original PR, this change is pretty hard to test due
to the fact that the behavior only occurs if stderr is connected to a
tty. I experimented with using pseudoterminals to test this but it's
still quite tricky due to the lack of x-platform non-blocking reads on
the pty.
<!-- How was it tested? -->
Fixes#11217
By default, a 64-bit uv does not see a 32-bit global (HKLM) installation
of Python in the registry
(https://github.com/astral-sh/uv/issues/11217). To work around this, we
manually request both 32-bit and 64-bit access using registry access
flags (https://peps.python.org/pep-0514/#sample-code). The flags have no
effect on 32-bit (https://stackoverflow.com/a/12796797/3549270).
This effect is that there is an asymmetry between discovery modes: For
the registry-based discovery using PEP 514, we discover both 32-bit and
64-bit Pythons, while for managed installations, we are stricter and
only discover those matching in bit-ness.
I tested this manually with an additional 32-bit installation of CPython
on a 64-bit machine and windows with 32-bit and 64-bit (x86_64 and i686)
builds of uv.
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.
In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
## Summary
Testing with `UV_PYTHON_INSTALL_DIR` environment variable has some
problems. This PR fix them.
- `UV_PYTHON_INSTALL_DIR` must be an absolute path.
- Cargo tries to find Python executables from each crates in test. If it
is relative path, cargo searches in different directories for each
tests.
- Skip the test asserting help messages.
- Clap shows the current value of the environment variables. If
`UV_PYTHON_INSTALL_DIR` is set, the test fails.
## Test Plan
<!-- How was it tested? -->
All tests pass with
`UV_PYTHON_INSTALL_DIR=/path/to/my/home/uv/target/testpython`.
## Summary
In. https://github.com/astral-sh/uv/issues/11857, we had a case of a
user that was seeing incorrect resolution results after upgrading to a
newer version of macOS, since we retained cache information about the
interpreter. This PR adds the OS name and version to the cache key for
the interpreter. This seems to be extremely cheap, and it's nice to make
this robust so that users don't run into the same confusion in the
future.
Closes https://github.com/astral-sh/uv/issues/11857.
## Summary
We need to decouple the "Is this shell supported by `update-shell`?"
logic from the "Does this shell have known configuration files?" logic,
specifically for Windows, which we can always update but not via
configuration files.
Closes https://github.com/astral-sh/uv/issues/11803.
## Summary
We use a similar strategy to the ephemeral overlay: set
`include-system-site-packages` in the `pyvenv.cfg`, and clear it
whenever we access a new environment.
Closes https://github.com/astral-sh/uv/issues/11829.
## Test Plan
Difficult to test because we don't really have support for system
packages in our test infrastructure. But...
```
> uv venv --system-site-packages
> ['', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python313.zip', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/lib-dynload', '/Users/crmarsh/.cache/uv/archive-v0/AhKcORkaCdbBl31VweRtG/lib/python3.13/site-packages', '/Users/crmarsh/workspace/uv/foo/.venv/lib/python3.13/site-packages', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/site-packages']
```
```
> uv venv
> ['', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python313.zip', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/lib-dynload', '/Users/crmarsh/.cache/uv/archive-v0/AhKcORkaCdbBl31VweRtG/lib/python3.13/site-packages', '/Users/crmarsh/workspace/uv/foo/.venv/lib/python3.13/site-packages']
```
Reworks how log verbosity flags work.
* `<no argument>` is the same, equivalent to `RUST_LOG=off`
* `-v` is the same, equivalent to `RUST_LOG=uv=debug`
* `-vv` is now equivalent to `RUST_LOG=uv=trace` (previously it only
enabled more log message context)
* `-vvv` is now equivalent to `RUST_LOG=trace` (previously it was
equivalent to `-vv`)
The "more context" that `-vv` had has been moved to an orthogonal
setting via an environment variable. Setting `UV_LOG_CONTEXT=1` will add
the extra context that `-vv` did.
In the future we may make these more granular as we try to use
`info!/warn!` more.
Fixes#1569
Fixes#11793
On Windows, trying to read a file inside what is not a directory but
another file results in a not found error, while on Unix we get a not a
directory error. We check explicitly if something included in a
workspace glob is a non-directory to fix the behavior on Windows.
Three network settings are always passed together (though in random
method parameter orders). I factored these out into a struct to make planned future
changes easier.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
This pull request introduces validation for unique index names in the
`tool.uv.index` field and adds corresponding tests to ensure the
functionality. The most important changes include adding a custom
deserializer function, updating the `ToolUv` struct to use the new
deserializer, and adding tests to verify the behavior.
Validation and deserialization:
*
[`crates/uv-workspace/src/pyproject.rs`](diffhunk://#diff-e12cd255985adfd45ab06f398cb420d2f543841ccbeea4175ccf827aa9215b9dR283-R311):
Added a custom deserializer function `deserialize_index_vec` to validate
that index names in the `tool.uv.index` field are unique.
*
[`crates/uv-workspace/src/pyproject.rs`](diffhunk://#diff-e12cd255985adfd45ab06f398cb420d2f543841ccbeea4175ccf827aa9215b9dR374):
Updated the `ToolUv` struct to use the `deserialize_index_vec` function
for the `index` field.
Testing:
*
[`crates/uv/tests/it/lock.rs`](diffhunk://#diff-82edd36151736f44055f699a34c8b19a63ffc4cf3c86bf5fb34d69f8ac88a957R15336):
Added a test `lock_repeat_named_index` to verify that duplicate index
names result in an error.
[[1]](diffhunk://#diff-82edd36151736f44055f699a34c8b19a63ffc4cf3c86bf5fb34d69f8ac88a957R15336)
[[2]](diffhunk://#diff-82edd36151736f44055f699a34c8b19a63ffc4cf3c86bf5fb34d69f8ac88a957R15360-R15402)
*
[`crates/uv/tests/it/lock.rs`](diffhunk://#diff-82edd36151736f44055f699a34c8b19a63ffc4cf3c86bf5fb34d69f8ac88a957R15360-R15402):
Added a test `lock_unique_named_index` to verify that unique index names
result in successful lock file generation.
Schema update:
*
[`uv.schema.json`](diffhunk://#diff-c669473b258a19ba6d3557d0369126773b68b27171989f265333a77bc5cb935bR205):
Updated the schema to set the default value of the `index` field to
`null`.
Fixes#11804
## Test Plan
### Steps to reproduce and verify the fix:
1. Clone the repository and checkout the feature branch
```bash
git clone https://github.com/astral-sh/uv.git
cd uv
git checkout feature/warn-duplicate-index-names
```
2. Build the modified binary
```bash
cargo build
```
3. Create a test project using the system installed uv
```bash
uv init uv-test
cd uv-test
```
4. Manually edit pyproject.toml to add duplicate index names
```toml
[[tool.uv.index]]
name = "alpha_b"
url = "<omitted>"
[[tool.uv.index]]
name = "alpha_b"
url = "<omitted>"
```
5. Try to add a package using the modified binary
```bash
../target/debug/uv add numpy
```
### Results
Before: use release binary
After: use self build binary
Now when attempting to use a pyproject.toml with duplicate index names,
the modified binary correctly detects the issue and produces an error
message:
```
error: Failed to parse: `pyproject.toml`
Caused by: TOML parse error at line 9, column 1
|
9 | [[tool.uv.index]]
| ^^^^^^^^^^^^^^^^^
duplicate index name `alpha_b`
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Follow up for https://github.com/astral-sh/uv/pull/11738
I missed this while reviewing the truncation changes.
`format!("{:.N}", value)` only truncates if the `fmt::Display`
implementation supports it (by reading `f.precision()` in trait
implementation).
So in our case `format!("{:.N}", version.to_string())` will work but not
`format!("{:.N}", version)` unless `Version` supports it.
Since we only need it once, I am just truncating after the string is
created.
## Test Plan
<!-- How was it tested? -->
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Closes#2410
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the name of files in `wheels` bucket to use a hash instead
of the wheel name as to not exceed maximum file length limit on various
systems.
This only addresses the primary concern of #2410. It still does _not_
address:
- Path limit of 260 on windows:
https://github.com/astral-sh/uv/issues/2410#issuecomment-2062020882
To solve this we need to opt-in to longer path limits on windows
([ref](https://github.com/astral-sh/uv/issues/2410#issuecomment-2150532658)),
but I think that is a separate issue and should be a separate MR.
- Exceeding filename limit while building a wheel from source
distribution
As per my understanding, this is out of uv's control. Name of the output
wheel will be decided by build-backend used by the project. For wheels
built from source distribution, pip also uses the wheel names in cache.
So I have not touched `sdists` cache.
I have added a `filename: WheelFileName` field in `Archive`, so we can
use it while indexing instead of relying on the filename on disk.
Another way to do this was to read `.dist-info/WHEEL` and
`.dist-info/METADATA` and build `WheelFileName` but that seems less
robust and will be slower.
## Test Plan
<!-- How was it tested? -->
Tested by installing `yt-dlp`, `httpie` and `sqlalchemy` and verifying
that cache files in `wheels` bucket use hash.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Currently, we're using both the official `windows-*` with
`windows-registry` crates as well as `winreg`, an older,
community-maintained crate.
To unify the codebase, we follow the lead of rustup that already
performed this migration
(bce3ed67d2).
This is also a prerequisite to unblock the unification of the
windows-sys crate versions.
I've manually tested that `uv tool update-shell` works for adding to
PATH and correctly detects when PATH was already added.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Resolves#11704
Propagate errors from `uv_console::confirm` up instead of `unwrap`ping
them, causing panics.
## Test Plan
<!-- How was it tested? -->
Regression testing the bug is very difficult, as the behavior of
`confirm` changes based on whether `uv` is talking to a `tty`. We can
trick it using ptys, but the best rust pty crate I could find only
provides blocking reads of the spawned child, which is insufficient to
write the regression test.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
The animation shadows any interactive authentication prompt which may
occur when resolving dependencies of private repos.
Fixes https://github.com/astral-sh/uv/issues/5107.
## Test Plan
I started creating `install_git_private_https_interactive` as a
regression test but am unsure how to test this because it is interactive
and I don't really know the test framework
## Summary
This is the pattern I see in a variety of crates, and I believe this is
preferred if you don't _need_ an owned `String`, since you can avoid the
allocation. This could be pretty impactful for us?
## Summary
Since we use `SmallString` internally, there's no benefit to passing an
owned string to the `PackageName` constructor (same goes for
`ExtraName`, etc.). I've kept them for now (maybe that will change in
the future, so it's useful to have clients passed own values if they
_can_), but removed a bunch of usages where we were casting from `&str`
to `String` needlessly to use the constructor.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
I noticed that when running "uv build --all-packages" in an empty
workspace with no buildable packages, it reports that there are
buildable packages. Which I believe to be an error in the message. This
patch fixes the typo. I did not find any relevant issues.
## Test Plan
I've verified, to the best of my ability, that this did not introduce
any additional errors in related existing tests. Considering the nature
of the change I believe it's sufficient.
## Summary
Like `uv add --script`, `uv lock --script` will now initialize a PEP 723
script tag if it doesn't already exist.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
* Upgrade the rust toolchain to 1.85.0. This does not increase the MSRV.
* Update windows trampoline to 1.86 nightly beta (previously in 1.85
nightly beta).
## Test Plan
Existing tests
## Summary
Likely not critical since these tend to run prior to resolution rather
than in parallel with it, but we _should_ respect in-flight requests
here.
According to the [UV
documentation](https://docs.astral.sh/uv/configuration/files/#env), the
UV_ENV_FILE environment variable should support multiple .env files,
separated by spaces. However, when I tried using this feature in my
repository, it didn’t work as expected.
To investigate, I checked the UV repository for relevant tests and found
`run_with_multiple_env_files`.
This test asserts the following `error: No environment file found at:
.env1 .env2.`
This discrepancy could indicate either a mismatch between the
documentation and the implementation or a bug in the code.
I decided to fix the issue in the code since the ability to pass
multiple `.env` files is a valuable feature.
If my fix isn’t appropriate, I’d be happy to make any necessary
adjustments.
---------
Co-authored-by: Yaroslav Limanskiy <yaroslav.limanskiy@pandadoc.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
I believe these are not necessary... They're currently used in two
places:
1. When building wheels. But that's already wrapped in an in-flight map,
which does the same thing.
2. When fetching source distribution metadata. But every route there
uses it's own `flock` to coordinate across processes, so this seems
redundant?
In #6827, we switched the uv-dev binary to not being built by default.
As an unintended side effect, we were also stopping to run the tests
that ensured the schema was up-to-date.
To fix this, we split uv-dev into an unconditional library, with only
the binary being a conditional build. This way, `cargo test` and `cargo
nextest` pick those tests up again.
An alternative would be running tests with the `dev` feature, with the
side effect of always building the uv-dev binary, too.
## Summary
As-is, we used the refined interpreter to _resolve_, but we then created
the tool environment with the "old" interpreter. So we risked running
(e.g.) code that requires Python 3.12 in a Python 3.10 environment. We
need to propagate the updated interpreter.
This is fairly hard to test, because it requires an environment in which
we're able to download new interpreters.
Closes
https://github.com/astral-sh/uv/issues/11678#issuecomment-2672659074.
Currently, `uv tool list --show-paths` will show backslashes as path
separators for packages but not entrypoints. This PR changes this to be
consistent.
Closes#10426.
## Summary
Today, if you have a lockfile that includes conflict markers, we write
those markers out to `requirements.txt` in `uv export`. This is
problematic, since no tool will ever evaluate those markers correctly
downstream.
This PR adds handling for the conflict markers, though it's quite
involved. Specifically, we have a new reachability algorithm that
tracks, for each node, the reachable marker for that node _and_ the
marker conditions under which each conflict item is `true` (at that
node).
I'm slightly worried that this algorithm could be wrong for graphs with
cycles, but we only use this logic for lockfiles with conflicts anyway,
so I think it's a strict improvement over the status quo.
Closes https://github.com/astral-sh/uv/issues/11559.
Closes https://github.com/astral-sh/uv/issues/11548.
## Summary
We need to compute the set of activated groups prior to evaluating the
conflict markers on the groups' dependencies.
Closes https://github.com/astral-sh/uv/issues/11648.
Solving spent a chunk of its time just converting resolutions, the left
two blocks:
These blocks are `ResolverOutput::from_state` with 1.3% and
`ForkState::into_resolution` with 4.1% of resolver thread runtime for
apache airflow universal.
We reduce the overhead spent in those functions, to now 1.1% and 2.1% of
resolver time spend in those functions by:
Commit 1: Replace the hash set for the edges with a vec in
`ForkState::into_resolution`. We deduplicate edges anyway when
collecting them, and the hash-and-insert was slow.
Commit 2: Reduce the distribution clonign in
`ResolverOutput::from_state` by using an `Arc`.
The same profile excerpt for the resolver with the branch (note that
there is now an unrelated block between the two we optimized):
Wall times are noisy, but the profiles show those changes as
improvements.
```
$ hyperfine --warmup 2 "./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal" "./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal"
Benchmark 1: ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 99.1 ms ± 3.8 ms [User: 111.8 ms, System: 115.5 ms]
Range (min … max): 93.6 ms … 110.4 ms 29 runs
Benchmark 2: ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 97.1 ms ± 4.3 ms [User: 114.8 ms, System: 112.0 ms]
Range (min … max): 90.9 ms … 112.4 ms 29 runs
Summary
./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal ran
1.02 ± 0.06 times faster than ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
```
Instead of always using all available threads for bytecode compilation,
respect `UV_CONCURRENT_INSTALLS`, so the parallelism is configurable
instead of hardcoded. We reuse the install limit since bytecode
compilation only runs after install.
Revert #11601 for now
We run Python interpreter discovery with `-I` (#2500) which means these
environments variables are ignored when determining `sys.path`. Unless
we decide to remove the `-I` flag from the `sys.path` query, we
shouldn't release these changes to interpreter discovery caching.
I noticed that the latest two `sync-python-releases` jobs failed due to
`httpx.RemoteProtocolError: peer closed connection without sending
complete message body (incomplete chunked read)`.
For the current python-build-standalone release, each request page
(defaulting to 30 items per page) takes about 20 seconds and loads
around 32MB of data. This extensive data load might be causing the
request to frequently fail.
In this PR, I reduced number of items per page to 10 and added
`Accept-Encoding: gzip, deflate` to the request header. Now, it takes
about 6 seconds to load, and the compressed response size has been
reduced to 534KB. I hope this would addresses the request failure.
We want to use `sys.path` for package discovery (#2500, #9849). For
that, we need to know the correct value of `sys.path`. `sys.path` is a
runtime-changeable value, which gets influenced from a lot of different
sources: Environment variables, CLI arguments, `.pth` files with
scripting, `sys.path.append()` at runtime, a distributor patching
Python, etc. We cannot capture them all accurately, especially since
it's possible to change `sys.path` mid-execution. Instead, we do a best
effort attempt at matching the user's expectation.
The assumption is that package installation generally happens in venv
site-packages, system/user site-packages (including pypy shipping
packages with std), and `PYTHONPATH`. Specifically, we reuse
`PYTHONPATH` as dedicated way for users to tell uv to include specific
directories in package discovery.
A common way to influence `sys.path` that is not using venvs is setting
`PYTHONPATH`. To support this we're capturing `PYTHONPATH` as part of
the cache invalidation, i.e. we refresh the interpreter metadata if it
changed. For completeness, we're also capturing other environment
variables documented as influencing `sys.path` or other fields in the
interpreter info.
This PR does not include reading registry values for `sys.path`
additions on Windows as documented in
https://docs.python.org/3.11/using/windows.html#finding-modules. It
notably also does not include parsing of python CLI arguments, we only
consider their environment variable versions for package installation
and listing. We could try parsing CLI flags in `uv run python`, but we'd
still miss them when Python is launched indirectly through a script, and
it's more consistent to only consider uv's own arguments and environment
variables, similar to uv's behavior in other places.
This change keeps dependency group keys sorted when adding new ones.
If earlier dependency group keys were not sorted, we just append the new
group key to avoid churn in `pyproject.toml`. See discussion on #11447.
I've added a new snapshot test to capture this case.
Closes#11447.
## Summary
When tests are run downstream, the `COLUMNS` environment variable is
used to force fixed output width and avoid test failures due to
different terminal widths. However, this occasionally causes test
regressions when other tests rely on different output width. Use the
same `COLUMNS` value in CI to ensure consistent output and catch any
regressions.
## Test Plan
It wasn't, it's supposed to be tested by the CI :-).
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
The particular example I honed in on here was the `e3nn -> sympy 1.13.1`
and `e3nn -> sympy 1.13.3` dependency edges. In particular, while the
former correctly has a conflict marker, the latter's conflict marker was
getting simplified to `true`. This makes the edges trivially
overlapping, and results in both of them getting installed
simultaneously. (A similar problem happens for the `e3nn -> torch`
dependency edges.)
Why does this happen? Well, conflict marker simplification works by
detecting which extras are known to be enabled (and disabled) for each
node in the graph. This ends up being expressed as a set of sets, where
each inner set contains items corresponding to "extras is included" or
"extra is excluded."
The logic then is if _all_ of these sets are satisfied by the conflict
marker on the dependency edge, then this conflict marker can be
simplified by assuming all of the inclusions/exclusions to be true.
In this particular case, we run into an issue where the set of
assumptions discovered for `e3nn` is:
{test[sevennet]}, {}, {~test[m3gnet], ~test[alignn], test[all]}
And the corresponding conflict marker for `e3nn -> sympy 1.13.1` is:
extra == 'extra-4-test-all'
or extra == 'extra-4-test-chgnet'
or (extra != 'extra-4-test-alignn' and extra != 'extra-4-test-m3gnet')
And the conflict marker for `e3nn -> sympy 1.13.3` is:
extra == 'extra-4-test-alignn' or extra == 'extra-4-test-m3gnet'
Evaluating each of the sets above for `sympy 1.13.1`'s conflict
marker results in them all being true. Simplifying in turn results in
the marker being true. For `sympy 1.13.3`, not all of the sets are
satisfied, so this marker is not simplified.
I think the fundamental problem here is that our inferences aren't quite
rich enough to make these logical leaps. In particular, the conflict
marker for `e3nn -> sympy 1.13.3` is not satisfied by _any_ of our sets.
One might therefore conclude that this dependency edge is impossible.
But! The `test[sevennet]` set doesn't actually rule out `test[m3gnet]`
from being included, for example, because there is no conflict. So it is
actually possible for this marker to evaluate to true.
And I think this reveals the problem: for the `e3nn -> sympy 1.13.1`
conflict marker, the inferences don't capture the fact that
`test[sevennet]` _might_ have `test[m3gnet]` enabled, and that would in
turn result in the conflict marker evaluating to `false`. This directly
implies that our simplification here is inappropriate.
It would be nice to revisit how we build our inferences here so that
they are richer and enable us to make correct logical leaps. For now, we
fix this particular bug with a bit of a cop-out: we skip conflict marker
simplification when there are ambiguous dependency edges.
Fixes#11479
The place to look in this snapshot is the `name = "e3nn"` dependency.
Its dependencies on `sympy` and `torch` consist of multiple versions
with overlapping conflict markers. They are getting incorrectly
simplified to `true`.
Initially, we were limiting Git schemes to HTTPS and SSH as only
supported schemes. We lost this validation in #3429. This incidentally
allowed file schemes, which apparently work with Git out of the box.
A caveat for this is that in tool.uv.sources, we parse the git field
always as URL. This caused a problem with #11425: repo = { git =
'c:\path\to\repo', rev = "xxxxx" } was parsed as a URL where c: is the
scheme, causing a bad error message down the line.
This PR:
* Puts Git URL validation back in place. It bans everything but HTTPS,
SSH, and file URLs. This could be a breaking change, if users were using
a git transport protocol were not aware of, even though never
intentionally supported.
* Allows file: URL in Git: This seems to be supported by Git and we were
supporting it albeit unintentionally, so it's reasonable to continue to
support it.
* It does not allow relative paths in the git field in tool.uv.sources.
Absolute file URLs are supported, whether we want relative file URLs for
Git too should be discussed separately.
Closes#3429: We reject the input with a proper error message, while
hinting the user towards file:. If there's still desire for relative
path support, we can keep it open.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Resolves#6913.
Add `tool.uv.build-constraint-dependencies` to pyproject.toml.
The changes are analogous to the constraint-dependencies feature
implemented in #5248.
Add documentation for `build-constraint-dependencies`
## Test Plan
Add tests for `uv lock`, `uv add`, `uv pip install` and `uv pip
compile`.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This typo wasn't caught because the `($arg:expr, false)` macro branch
was never exercised.
For example, prior to this change, if you add
```
show_settings!(globals, false);
```
below, you'll get a compiler error.
When running `uv pip install .` in a directory with a pyproject.toml
that does not configure a build, we will invoke setuptools and get a
wheel we can't parse (https://github.com/astral-sh/uv/issues/11344).
This PR adds warnings around these setups.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
## Summary
This PR fixes a subtle issue arising from our propagation of
preferences. When we resolve a fork, we take the solution from that fork
and mark all the chosen versions as "preferred" as we move on to the
next fork.
In this specific case, the resolver ended up solving a macOS-specific
fork first, which led us to pick `2.6.0` rather than `2.6.0+cpu`. This
in itself is correct; but when we moved on to the next fork, we
preferred `2.6.0` over `2.6.0+cpu`, despite the fact that `2.6.0` _only_
includes macOS wheel, and that branch was focused on Linux.
Now, in preferences, we prefer local variants (if they exist). If the
local variant ends up not working, we'll presumedly backtrack to the
base version anyway.
Closes https://github.com/astral-sh/uv/issues/11406.
## Summary
If the user provides a PEP 508 requirement (e.g., `uvx
change_wheel_version`), then we should us that verbatim for the
executable, rather than normalizing the package name.
Closes https://github.com/astral-sh/uv/issues/11521.
## Summary
This PR revives https://github.com/astral-sh/uv/pull/10017, which might
be viable now that we _don't_ enforce any platforms by default.
The basic idea here is that users can mark certain platforms as required
(empty, by default). When resolving, we ensure that the specified
platforms have wheel coverage, backtracking if not.
For example, to require that we include a version of PyTorch that
supports Intel macOS:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = ["torch>1.13"]
[tool.uv]
required-platforms = [
"sys_platform == 'darwin' and platform_machine == 'x86_64'"
]
```
Other than that, the forking is identical to past iterations of this PR.
This would give users a way to resolve the tail of issues in #9711, but
with manual opt-in to supporting specific platforms.
## Summary
This is an alternative to the approach we took in #11063 whereby we
always included `provides-extra` and `requires-dist`, since we needed
some way to differentiate between "no extras" and "lockfile was
generated by a uv version that didn't include extras".
Instead, this PR adds a minor version (called a "revision") to the
lockfile that we can use to indicate support for this feature. While
lockfile version bumps are backwards-incompatible, older uv versions
_can_ read lockfiles with a later revision -- they just won't understand
all the data.
In a future major version bump, we could simplify things and change the
schema to use a (major, minor) format instead of these two separate
fields. But this is the only way to do it that's backwards-compatible
with existing uv versions.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Initially it seemed like `app.py` might be slightly more desirable but
people seem to overwhelmingly favour `main.py` as a good "generic" name.
Fixes#7782
Closes#11285
Closes https://github.com/astral-sh/uv/pull/11437
This changes `-p` from an alias of `--python-version` to `--python`
while retaining backwards compatibility for `--python-version`-like
fallback behavior when the requested version, e.g., `-p 3.12`, cannot be
found.
This was initially implemented with a hidden `--python-legacy` flag
which allows us to special case the short `-p` flag — unlike the
implementation in #11437. However, after further discussion, we decided
the behavior difference between `-p` and `--python` would be confusing
so now `-p` is an alias for `--python` and `--python` is special-cased
when a version is used.
Additionally, we now respect the `UV_PYTHON` environment variable, but
it is ignored when `--python-version` is set. If you want different
`--python-version` and `--python` values, you must do so explicitly. I
considered banning this, but it is valid for e.g. `--python pypy
--python-version 3.12`
Unlike https://github.com/astral-sh/uv/pull/10222, this does not respect
`UV_PYTHON` in `uv python uninstall` (continuing to require an explicit
target there) which I think is simpler and matches our `.python-version`
file behavior.
---------
Co-authored-by: Choudhry Abdullah <cabdulla@trinity.edu>
Co-authored-by: Choudhry Abdullah <choudhry347@choudhrys-air-2.trinity.local>
Co-authored-by: Aria Desires <aria.desires@gmail.com>
Closes#10597.
Recreated https://github.com/astral-sh/uv/pull/10925 that got closed as
the base branch got merged.
Snapshot tests.
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
`uv publish` has not changed for some time, it has [notable production
usage](https://github.com/search?q=%22uv+publish%22&type=code) and there
are no outstanding blockers, it is time to stabilize it with the 0.6
release.
Publishing is only usable through `uv publish`. You need to build source
distributions and wheels ahead of time, usually with `uv build`.
By default, `uv publish` will upload all source distributions and wheels
in the `dist/` folder, ignoring all non-matching filenames. By default,
`uv build` and most other build frontend write their artifacts to
`dist/`. Together, we can build a publish workflow including a smoke
test that all relevant files have actually been included in the wheel:
```
uv build
uv venv
uv pip install --find-links dist ...
uv run smoke_test.py
uv publish
```
There are 3 options supported in configuration files:
- `tool.uv.publish-url`
- `tool.uv.trusted-publishing`
- `tool.uv.check-url`
Options support on the CLI and through environment variables for index
configuration:
```
--index <INDEX>
The name of an index in the configuration to use for publishing [env: UV_PUBLISH_INDEX=]
--publish-url <PUBLISH_URL>
The URL of the upload endpoint (not the index URL) [env: UV_PUBLISH_URL=]
--check-url <CHECK_URL>
Check an index URL for existing files to skip duplicate uploads [env: UV_PUBLISH_CHECK_URL=]
```
There are two ways to configure `uv publish`: Passing options
individually or using the index API.
For the individual options, there `--publish-url` and `--check-url`, and
their configuration counterparts, `tool.uv.publish_url` and
`tool.uv.check_url`. `--publish-url` is named this way to be clearly
different from the simple index URL, since uploading to the index URL
leads to unclear errors, or worse a 200 OK with no effect. While we
intend to keep supporting this configuration, the index API is better
integrated.
In the index API, the user specifies `[[tool.uv.index]]`, with an index
name, the simple index URL and the publish URL. The `publish-url` and
`url` are equivalent to `--publish-url` and `--check-url`. The `url`
being mandatory makes for a better upload behavior (next paragraph).
```toml
[[tool.uv.index]]
name = "pypi"
url = "https://pypi.org/simple"
publish-url = "https://upload.pypi.org/legacy/"
```
A version of a package contains multiple files, for pure-python packages
usually a source distribution and a wheel, for native packages usually
many, larger wheels and a source distributions. Uploads in the not
officially specified Upload API 1.0 are file based: Once you upload a
file, the version is created, even though most files are still missing.
When uploading a series of files fails in the middle (e.g. the CI server
breaks), the release is only half uploaded. For such cases, you want to
re-try the upload. The response of an index when re-uploading a file is
implementation defined. Notably, PyPI accepts uploads of the same file
again with status 200, but rejects uploads of a file with the same name
but different contents with status 400. Other indexes reject all
attempts at re-uploads with different status codes and messages. Twine
handles this with `--skip-existing`, which allows ignoring errors due to
files with the same name as an existing file being uploaded, however
this does also not error when uploading a file with different contents
but the same name, which indicates a problem with the publish pipeline.
To properly solve this, we need the ability to stage releases: Files of
a version are uploaded to a staging area, and only when all files are
uploaded, we atomically publish the release. When an upload breaks or CI
fails, we can discard or overwrite the staging area and try again. This
will only be properly solved by PEP 694 "Upload 2.0 API for Python
Package Indexes", with unclear progress. For local publishing, it would
also be convenient to be able to check which files exist and what their
hashes are from only the publish URL, so files in the `dist/` folder
from a previous release can be ignored.
In the Upload API 1.0, we need to upload transformed METADATA fields
along with the file as form-data. We currently upload only recognized
metadata fields, where we know how to translate the field name to the
form-data name. This means when a user adds unknown, wrong or future-PEP
metadata we miss it. To me best knowledge no index currently verifies
that the form-data and the METADATA file in the wheel match.
Upload API 2.0 will be an entirely new protocol. It is unclear how we
will decide whether to use Upload API 1.0 or Upload API 2.0 once the
latter is released. Upload API 2.0 will remove the need for a check URL.
This means no changes for `--index`, but `--check-url` will be
incompatible with Upload API 2.0.
Options support on the CLI and through environment variables for
authentication:
```
-u, --username <USERNAME>
The username for the upload [env: UV_PUBLISH_USERNAME=]
-p, --password <PASSWORD>
The password for the upload [env: UV_PUBLISH_PASSWORD=]
-t, --token <TOKEN>
The token for the upload [env: UV_PUBLISH_TOKEN=]
--trusted-publishing <TRUSTED_PUBLISHING>
Configure using trusted publishing through GitHub Actions [possible values: automatic, always,
never]
--keyring-provider <KEYRING_PROVIDER>
Attempt to use `keyring` for authentication for remote requirements files [env:
UV_KEYRING_PROVIDER=] [possible values: disabled, subprocess]
```
We need credentials for the publish URL, and we may need credentials for
the check URL.
We support credentials from environment variables, the CLI, the URL, the
keyring, trusted publishing or a prompt.
The username can come from, in order:
- Mutually exclusive:
- `--username` or `UV_PUBLISH_USERNAME`. The CLI option overrides the
environment variable
- The username field in the publish URL
- If `--token` or `UV_PUBLISH_TOKEN` are used, it is `__token__`. The
CLI option overrides the environment variable
- If trusted publishing is available, it is `__token__`
- (We currently do not read the username from the keyring)
- If stderr is a tty, prompt the user
The password can come from, in order:
- Mutually exclusive:
- `--password` or `UV_PUBLISH_PASSWORD`. The CLI option overrides the
environment variable
- The password field in the publish URL
- If `--token` or `UV_PUBLISH_TOKEN` are used, it is the token value.
The CLI option overrides the environment variable
- If the keyring is enabled, the keyring entry for the URL and username
- If trusted publishing is available, the trusted publishing token
- If stderr is a tty, prompt the user
If no credentials are found, we do a final check in the auth middleware
cache and otherwise error without sending the request.
Trusted publishing is only supported in GitHub Actions. By default, we
try to retrieve a token from it in GitHub Actions (`GITHUB_ACTIONS` is
`true`) but continue even it this fails. Trusted publishing can be
forced with `--trusted-publishing always`, to error on misconfiguration,
or deactivated with `--trusted-publishing never`. The option can also be
configured through `tool.uv.trusted-publishing`.
When `--check-url` or `--index` are used, we may need credentials for
the index URL, too. These are handle separately by the same rules as
using the index anywhere else. The `--keyring-provier` option is however
shared between them, turning the keyring on for either turns it on for
both.
As future option, we could read `UV_INDEX_USERNAME` and
`UV_INDEX_PASSWORD` as fallbacks for the publish credentials
(https://github.com/astral-sh/uv/issues/9845). This however would clash
with prompting: When index credentials and upload credentials are not
the same (they usually should be different, since regular uv operations
should have less privileges than publish), we would then instead of
prompting use the wrong credentials from `UV_INDEX_*` and fail.
A major UX problem is that there is no standard for the username when
using a token (or rather, there is no standard for just sending a token
without a username). PyPI uses `__token__`, Cloudsmith used to use your
username or `token`, but now also supports `__token__`
(https://github.com/astral-sh/uv/issues/8221), while Google Cloud
Artifacts always uses `oauth2accesstoken`
(https://github.com/astral-sh/uv/issues/9778). This means the index
documentation may say you're getting a token for authentication, but you
must not use `--token`, you must instead set username and password. This
is something that we can hopefully fix with Upload API 2.0.
An unsolved problem with the keyring is that you it's best practice to
use publish tokens scoped to projects and store tokens in a secure
location such as the keyring, but the keyring saves a single password
per publish URL and username combination. That means that it can't
natively store separate passwords for publishing multiple packages. The
current hack around this is using the package name as query parameter,
e.g. `https://test.pypi.org/legacy/?astral-test-keyring`, as PyPI
ignores this query parameter. This is however only applicable when
publishing locally and not from CI.
Another problem is that the keyring implementation currently relies on
the `keyring` pypi package, which needs to be installed in PATH together
with its plugins and is comparatively slow. This would be improved by
native keyring support (https://github.com/astral-sh/uv/issues/10867),
with the same caveats such as keyring plugins that shared with the
simple index API.
We currently don't upload attestations (PEP 740). Attestations are an
additional field in the form-data, so we should be able to add them
transparently without any changes to the API, unless we want to add a
switch to deactivate even when trusted publishing is used. See also
https://trailofbits.github.io/are-we-pep740-yet/.
Setuptools is writing an invalid combination of Metadata-Version and
used metadata fields in some cases, which PyPI correctly rejects
(https://github.com/astral-sh/uv/issues/9513).
We set a 15min overall timeout since reqwest is missing a write timeout
option (https://github.com/seanmonstar/reqwest/issues/2403).
https://github.com/astral-sh/uv/issues/8641 and
https://github.com/astral-sh/uv/issues/8774: We build artifact checking
in some capacity. This should be done ideally by the build backend or at
latest as part of `uv build`, doing it as part of publish is too late.
Closes#7839
---
Let me know if i missed anything.
We added this to help with resolving some specific packages, and for
parity with Poetry. But in some cases, this metadata is just wrong, and
at the very least it's unreliable.
Closes https://github.com/astral-sh/uv/issues/8989.
Closes#10945.
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.
Closes#11263.
We've never bumped the version of this bucket, and we may never do so...
But it's still incorrect for us to omit it from these serialized structs
in the cache. Specifically, these structs include a pointer into the
archive bucket (namely, the ID). But we don't include the bucket
version! So, in theory, we could end up pointing to archives that don't
match the current bucket version expected in the code.
## Summary
Just a logic issue... If we see a dynamic field that isn't `"version"`,
we end up _not_ propagating the fact that `"version"` is also dynamic.
Closes https://github.com/astral-sh/uv/issues/11460.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Handle potential infinite recursion if `uv run` recursively invokes `uv
run`. This can happen if the shebang line of a script includes `uv run`,
but does not pass `--script`.
Handled by adding a new environment variable `UV_RUN_RECURSION_DEPTH`,
which contains a counter of the number of times that uv run has been
recursively invoked. If unset, it defaults to zero, and each time uv run
starts a subprocess we increment the counter, erroring if the value is
greater than a configurable (but not currently exposed or documented)
threshold.
Closes https://github.com/astral-sh/uv/issues/11220.
## Test Plan
I've added a snapshot test to `uv/crates/uv/tests/it/run` that tests the
end-to-end recursion detection flow. I've currently made it a unix-only
test because I'm not sure offhand how uv run will interact with shebang
lines on windows.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Closes https://github.com/astral-sh/uv/issues/11288
I tested the reproduction there manually.
I'm a little uncertain about this behavior, it's not true to the spirit
of `--python <dir>` selecting a target environment but this method is
only used to see if an existing environment matches for the purpose of
invalidation in projects and tools where I think we always force a
separate environment anyway?
## Summary
We need to add indexes in the order in which they're respected by the
resolver. Otherwise, we risk writing an index to the `pyproject.toml`
that is canonically equal (but not verbatim equivalent) to the index we
use during resolutin.
Closes https://github.com/astral-sh/uv/issues/11312.
## Summary
The environment is located at a stable path within the cache, based on
the script's absolute path.
If a lockfile exists for the script, then we use our standard lockfile
semantics (i.e., update the lockfile if necessary, etc.); if not, we
just do a `uv pip sync` (roughly).
Example usage:
```
❯ uv init --script hello.py
Initialized script at `hello.py`
❯ uv add --script hello.py requests
Updated `hello.py`
❯ cargo run sync --script hello.py
Using script environment at: /Users/crmarsh/.cache/uv/environments-v1/hello-84e289fe3f6241a0
Resolved 5 packages in 3ms
Installed 5 packages in 12ms
+ certifi==2025.1.31
+ charset-normalizer==3.4.1
+ idna==3.10
+ requests==2.32.3
+ urllib3==2.3.0
```
Closes https://github.com/astral-sh/uv/issues/6637.
## Summary
This is a follow-on to #11347 to use a stable directory for remote and
stdin scripts. The annoying piece here was figuring out what to use as
the cache key. For remote scripts, I'm using the URL; for stdin scripts,
there isn't any identifying information, so I'm just using a hash of the
metadata.
## Summary
Today, scripts use `CachedEnvironment`, which results in a different
virtual environment path every time the interpreter changes _or_ the
project requirements change. This makes it impossible to provide users
with a stable path to the script that they can use for (e.g.) directing
their editor.
This PR modifies `uv run` to use a stable path for local scripts (we
continue to use `CachedEnvironment` for remote scripts and scripts from
`stdin`). The logic now looks a lot more like it does for projects: we
`get_or_init` an environment, etc.
For now, the path to the script is like:
`environments-v1/4485801245a4732f`, where `4485801245a4732f` is a SHA of
the absolute path to the script. But I'm not picky on that :)
## Summary
This PR refactors the whole `Target` abstraction, mostly to remove all
the repeated `From*` variants and logic in favor of a higher-level
struct that captures those details separately.
In doing so, it also adds support to `uvx` for unnamed requirements, for
parity with `uv tool install`. So, e.g., the following will show the
`flask` version:
```
uvx git+https://github.com/pallets/flask --version
```
I think this makes sense conceptually since we already support arbitrary
named requirements there.
## Summary
Right now, `uv sync --dry-run` returns the interpreter that _would've_
been used to create the environment; so we end up using _that_
interpreter's `site-packages`, and return changes relative to that
interpreter. Instead, we now create a temporary virtual environment and
compare against that.
Closes https://github.com/astral-sh/uv/issues/11422.
## Summary
We currently enforce that if you do `uv pip install
./dist/iniconfig-1.0.0.tar.gz`, the build _must_ produce a wheel like
`iniconfig-1.0.0-py3-none-any.whl` (i.e., the name and version must
match). It turns out some packages produce a wheel that has a local
suffix on it, like `vllm`. This PR makes the check a little more
permissive in that we now accept `1.0.0` or that version with a local
suffix (e.g., `1.0.0+cpu`). I don't love this practice, but we already
relaxed this check when _installing_ a wheel, so this seems reasonable:
5e15881dcc/crates/uv-install-wheel/src/install.rs (L50-L52)
Note that this is _still_ stricter than pip. pip seems to only require
that the package name is the same (i.e., `iniconfig` matches
`iniconfig`; but they'll happily install a wheel like
`iniconfig-2.0.0-py3-none-any.whl` given
`./dist/iniconfig-1.0.0.tar.gz`).
Closes https://github.com/astral-sh/uv/issues/11038.
A user reported a homebrew Python that would raise an exception in the
interpreter probing script because `platform.mac_ver()` returned `('',
('', '', ''), '')` on his installation due to
https://github.com/Homebrew/homebrew-core/issues/206778
This is easy enough to catch and show a proper error message instead of
the Python backtrace.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
This adds `NO_BINARY` and `NO_BINARY_PACKAGE` environment variables to
the uv CLI, allowing the user to specify packages to build from source
using environment variables. Its not a complete fix for #4291 as it does
not handle the `pip` subcommand.
## Test Plan
This was tested by running `uv sync` with various `UV_NO_BINARY` and
`UV_NO_BINARY_PACKAGE` environment variables set and checking that the
correct set of packages were compiled rather than taken from pre-built
wheels.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Now that `version` is an optional field, we shouldn't error if an
unambiguous package is lacking a version. We can still enforce the same
guarantees via `source`, since we always set version and source
together, if the package is unambiguous. I also retained the same error
for non-local packages that lack a version like this.
Closes https://github.com/astral-sh/uv/issues/11384.
The underlying cause here, I believe, was that we weren't accounting
for the case where an edge could be visited *without* any extras
enabled. Because of that, we got into situations where we thought
there was only one path to an edge when there were actually more
paths. This in turn lead to us erroneously doing simplification where
it actually isn't justified. And in turn lead to duplicate versions
of the same package being installed in the same environment.
The fix for this ends up being really simple: in the case where we
don't add any conflict items for a package during graph traversal,
we materialize an empty set of conflicts to mark the case of no
extras being enabled when visiting the child edges. This is enough
to propagate the knowledge of multiple paths to the same edge and
causes us to avoid doing improper simplifications.
This does fix the problem in the snapshot, but it does also I think
lead to other cases where simplifications are no longer possible
(hence the changes to the airflow snapshot). But this seems
expected, since we are doing strictly less simplification than we
were before. It's unclear if all of those cases were actual bugs
or not though.
The snapshot is too big to meaningfully read, but the problem is
in the dependencies of `torchmetrics`:
[[package]]
name = "torchmetrics"
version = "1.6.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "lightning-utilities" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "torch", version = "2.2.1", source = { registry = "https://pypi.org/simple" } },
{ name = "torch", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-4-test-chgnet' or extra != 'extra-4-test-m3gnet'" },
]
The conflict markers here are overlapping, which means
both can be included in the same environment.
Previously, we patched pkg-config .pc files to have the absolute path to
the directory where we unpack a python-build-standalone release. As
discussed in #11028, we can use ${pcfiledir} in a .pc file to indicate
paths relative to the location of the file itself.
This change was implemented in astral-sh/python-build-standalone#507, so
for newer python-build-standalone releases, we don't need to do any
patching. Optimize this case by only modifying the .pc file if an actual
change is needed (which might be helpful down the line with hard links
or something). For older releases, change uv's patch to match what
python-build-standalone now does.
## Motivation
No-op `uv lock` in apache airflow
(891c67f210ab7c877d1f00ea6ea3d3cdbb0e96ef) is slow, which makes `uv run`
slow, too.
Reference project:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 16.3 ms ± 1.5 ms [User: 9.8 ms, System: 6.4 ms]
Range (min … max): 13.0 ms … 20.0 ms 186 runs
```
Apache airflow before:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 161.0 ms ± 5.2 ms [User: 135.3 ms, System: 24.1 ms]
Range (min … max): 155.0 ms … 176.3 ms 18 runs
```
## Optimization
`FlatRequiresDist::from_requirements` is taking 50% of main thread
runtime.
Before:
After both commits:
Apache airflow after the first commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 122.3 ms ± 5.4 ms [User: 96.1 ms, System: 24.7 ms]
Range (min … max): 114.0 ms … 133.2 ms 23 runs
```
Apache airflow after the second commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 108.5 ms ± 3.4 ms [User: 83.2 ms, System: 24.2 ms]
Range (min … max): 103.6 ms … 119.9 ms 28 runs
```
## Summary
These are used for coordination across processes. If you run uv under,
e.g., the root user, then under a different user, I don't think we
should prevent you from acquiring the lock.
Closes https://github.com/astral-sh/uv/issues/11324.
Includes https://pypy.org/posts/2025/02/pypy-v7318-release.html
These are labeled as betas in the post but not anywhere obvious to me?
I'm not sure we need to portray this to users.
Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
## Summary
No behavior changes... This just separates the formatting from the
collection of the results, and also fixes a bug whereby we didn't say
"No changes detected" in some cases.
Given an input in the shape:
```
foo[bar]==1.0.0; sys_platform == 'linux'
foo==1.0.0; sys_platform != 'linux'
```
We would write either
```
foo==1.0.0; sys_platform == 'linux'
```
or
```
foo==1.0.0
```
depending on the iteration order, as the first one is from the marker
proxy package and the second one from the package without marker.
The fix correctly merges graph entries when there are two nodes with
different extras and different markers.
I tried to write a packse test but it failed due to a different
iteration order showing the correct case directly instead of the failing
one we'd need.
Only `strip_extras` is affected, since `combine_extras` uses
`version_marker`.
When stderr is not a tty, we currently don't show any messages for build
or large downloads, since indicatif is hidden. We can improve this by
showing a message for:
* Starting and finishing a large download (>1MB)
* Starting and finishing a build
Downloads are limited to 1MB or unknown size to keep the logs concise
and not scroll the entire terminal away for a download that finishes
almost immediately.
These messages are not captured in the tests since their order is
non-deterministic (downloads and builds race to finish).
There are no "tick" messages for large downloads yet, we could e.g. show
an update on runnning downloads every n seconds.
Part of #11121
**Test Plan**
```
$ uv venv && FORCE_COLOR=1 cargo run -q pip install numpy --no-binary :all: --no-cache 2>&1 | tee a.txt
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 1 package in 221ms
Building numpy==2.2.2
Built numpy==2.2.2
Prepared 1 package in 2m 34s
Installed 1 package in 6ms
+ numpy==2.2.2
```
```
$ uv venv && FORCE_COLOR=1 cargo run -q pip install torch --no-cache 2>&1 | tee b.txt
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 24 packages in 648ms
Downloading setuptools (1.2MiB)
Downloading nvidia-cuda-cupti-cu12 (13.2MiB)
Downloading torch (731.1MiB)
Downloading nvidia-nvjitlink-cu12 (20.1MiB)
Downloading nvidia-cufft-cu12 (201.7MiB)
Downloading nvidia-cuda-nvrtc-cu12 (23.5MiB)
Downloading nvidia-curand-cu12 (53.7MiB)
Downloading nvidia-nccl-cu12 (179.9MiB)
Downloading nvidia-cudnn-cu12 (634.0MiB)
Downloading nvidia-cublas-cu12 (346.6MiB)
Downloading sympy (5.9MiB)
Downloading nvidia-cusparse-cu12 (197.8MiB)
Downloading nvidia-cusparselt-cu12 (143.1MiB)
Downloading networkx (1.6MiB)
Downloading nvidia-cusolver-cu12 (122.0MiB)
Downloading triton (241.4MiB)
Downloaded setuptools
Downloaded networkx
Downloaded sympy
Downloaded nvidia-cuda-cupti-cu12
Downloaded nvidia-nvjitlink-cu12
Downloaded nvidia-cuda-nvrtc-cu12
Downloaded nvidia-curand-cu12
[...]
```
## Summary
This is attempting to solve the same problem surfaced in #11208 and
#11209. However, those PRs only worked for our own managed Pythons. In
Gentoo, for example, they disable the managed Pythons, which led to
failures in the test suite, because the "base Python" returned after
creating a virtual environment would differ from the "base Python" that
you get after _querying_ an existing virtual environment.
The fix here is to apply our same base Python normalization and
discovery logic, to non-standalone / non-managed Pythons. We continue to
use `sys._base_executable` for such Pythons when creating the
virtualenv, but when _caching_, we perform this second discovery step.
Closes https://github.com/astral-sh/uv/issues/11237.
This is a rewrite of the groups subsystem to have more clear semantics,
and some adjustments to the CLI flag constraints. In doing so, the
following bugs are fixed:
* `--no-default-groups --no-group foo` is no longer needlessly rejected
* `--all-groups --no-default-groups` now correctly evaluates to
`--all-groups` where previously it was erroneously being interpretted as
just `--no-default-groups`
* `--all-groups --only-dev` is now illegal, where previously it was
accepted and mishandled, as if it was a mythical `--only-all-groups`
flag
Fixes#10890Closes#10891
## Summary
If you `uv run` from the same directory via multiple processes at the
same time, some of them will fail as they'll see an "incomplete" virtual
environment.
Closes https://github.com/astral-sh/uv/issues/11219.
I think `UV_PROJECT_ENVIRONMENT` is too complicated for use-cases where
the user wants to sync to the active environment. I don't see a
compelling reason not to make opt-in easier. I see a lot of questions
about how to deal with this warning in the issue tracker, but it seems
painful to collect them here for posterity.
A notable behavior here — we'll treat this as equivalent to
`UV_PROJECT_ENVIRONMENT` so... if you point us to a valid virtual
environment that needs to be recreated for some reason (e.g., new Python
version request), we'll happily delete it and start over.
## Summary
This PR removes the ephemeral `.pth` overlay when using a cached
environment. This solution isn't _completely_ safe, since we could
remove the `.pth` file just as another process is starting the
environment... But that risk already exists today, since we could
_overwrite_ the `.pth` file just as another process is starting the
environment, so I think what I've added here is a strict improvement.
Ideally, we wouldn't write this file at all, and we'd instead somehow
(e.g.) pass a file to the interpreter to run at startup? Or find some
other solution that doesn't require poisoning the cache like this.
Closes https://github.com/astral-sh/uv/issues/11117.
# Test Plan
Ran through the great reproduction steps from the linked issue.
Before:
After:
## Summary
I'm not sure that this has much of an effect in practice, but currently,
when we return a virtual environment, the `sys_base_executable ` of the
parent ends up being retained as `sys_base_executable` of the created
environment. But these can be, like, subtly different? If you have a
symlink to a Python, then for the symlink, `sys_base_executable` will be
equal to `sys_executable`. But when you create a virtual environment for
that interpreter, we'll set `home` to the resolved symlink, and so
`sys_base_executable` will be the resolved symlink too, in general.
Anyway, this means that we should now have a consistent value between
(1) returning `Virtualenv` from the creation routine and (2) querying
the created interpreter.
## Summary
It turns out that we were returning slightly different interpreter paths
on repeated `uv run --with` commands. This likely didn't affect many (or
any?) users, but it does affect our test suite, since in the test suite,
we use a symlinked interpreter.
The issue is that on first invocation, we create the virtual
environment, and that returns the path to the `python` executable in the
environment. On second invocation, we return the `python3` executable,
since that gets priority during discovery. This on its own is
potentially ok. The issue is that these resolve to different
`sys._base_executable` values in these flows... The latter gets the
correct value (since it's read from the `home` key), but the former gets
the incorrect value (since it's just the `base_executable` of the
executable that created the virtualenv, which is the symlink).
We now use the same logic to determine the "cached interpreter" as in
virtual environment creation, to ensure consistency between those paths.
Closes https://github.com/astral-sh/uv/issues/11214
Special-cases the first Python executable we find on the `PATH`,
allowing it to be considered during searches for virtual environments.
For some context, there are two stages to Python interpreter discovery
1. We find possible Python executables in various sources
2. We query the executables to determine canonical metadata about the
interpreter
We can't really be "sure" if an executable is a complaint virtual
environment during (1), we need to query the interpreter first. This
means that if you're only allowed to installed into virtual
environments, we'll query every interpreter on your PATH. This is not
performant, and causes confusion for users. Notably, I recently improved
error messaging when we can't find any valid interpreters, by showing
the error message we encounter while querying an interpreter (if any).
However, this is problematic when there's an error for an interpreter
that is not relevant to your search. In
https://github.com/astral-sh/uv/pull/11143, I added filtering to avoid
querying additional interpreters, but that regressed some user
experiences where they were relying on us finding implicitly active
virtual environments via the PATH.
In https://github.com/astral-sh/tokio-tar/pull/2, we accidentally
changed the `target_base` from the target base to the parent of the
file. This would cause hardlink unpacking to fail.
Example: A hardlink at `hardlinked-0.1.0/pyproject.toml` pointing to
`hardlinked-0.1.0/pyproject.toml.real` would try pointing to
`hardlinked-0.1.0/hardlinked-0.1.0/pyproject.toml.real` instead and fail
the unpacking.
The actual fix is in astral-tokio-tar, on the uv side there are only tests.
Fixes#11213
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
I got a bit confused when testing `[dependency-groups]` because uv's
error message had the same typo I did in my `pyproject.toml`.
I tried to fix it, as well as a few comment I found along the way.
These are noisy relative to the effect they have on the user. It seems
better to prioritize hints on poor resolutions. Notably, it seems hard
to make these "not noisy" ref #11091.
Does not include the "lowest" resolution mode, in which lower bounds are
critical.
With the parallel simple index fetching, we would only acquire one
download concurrency token, meaning that we could in the worst case make
times the number of indexes more requests than the user requested limit.
We fix this by passing the semaphore down to the simple API method.
Looks like the set based prioritize tracking from
https://github.com/pubgrub-rs/pubgrub/pull/313 is a slight speedup.
I assume the changed derivation tree in the error snapshot is due to
out-of-sync virtual package priorities, while the main package priority
defining the solution remains stable.
```
$ hyperfine --warmup 2 "./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal" "./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal"
Benchmark 1: ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 115.0 ms ± 4.8 ms [User: 131.0 ms, System: 113.6 ms]
Range (min … max): 108.1 ms … 125.8 ms 25 runs
Benchmark 2: ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 105.4 ms ± 2.6 ms [User: 118.5 ms, System: 113.5 ms]
Range (min … max): 101.1 ms … 111.9 ms 28 runs
Summary
./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal ran
1.09 ± 0.05 times faster than ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
```
uv-install-wheel had the logic for laying out the installation and for
linking a directory in the same module. We split them up to isolate each
module's logic and tighten the crate's interface to only expose top
level members.
No logic changes, only moving code around.
As before, these are fine-grained PATs and will expire in 366 days.
They're generated by splitting the token into three parts (by `_`) and
base64 encoding.
## Summary
This lets us drop a dependency entirely. `percent-encoding` is used by
`url` and so is already in the graph, whereas `urlencoding` isn't used
by anything else.
## Summary
This PR adds an additional normalization step to `CanonicalUrl` whereby
we now percent-decode the path, to ensure that (e.g.)
`torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` and
`torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` are considered
equal. Further, when generating the "reinstall" report, we use the
canonical URL rather than the verbatim URL.
In making this change, I also learned that we don't apply any of the
normalization passes to `file://` URLs. I inadvertently removed it in
93d606aba2,
since setting the password or URL on ` file://` URL errors -- but now
suppress those errors anyway.
Closes https://github.com/astral-sh/uv/issues/11082.
## Test Plan
- Downloaded a [PyTorch
wheel](https://download.pytorch.org/whl/cpu-cxx11-abi/torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl)
- `python3.9 -m pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --platform
linux_x86_64 --target foo --no-deps`
- `cargo run pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --python-platform
linux --python-version 3.9 --target foo --no-deps`
- Verified that the package had the `~` symbol for the reinstall.
konsti
Zanie Blue
github-actions[bot]
leiserfg
Charlie Marsh
Christopher Tee
Meitar Reihan
Aria Desires
renovate[bot]
justin
Jo
Tom Boshoven
John Mumm
Nozomi R
Zohar Meir
Merlin
Nick Janetakis
Ahmed Ilyas
InSync
Christian Bager Bach Houmann
James Z.M. Gao
samypr100
Tim Felgentreff
Jean-Michel Rouet
JonCholas
Christian Sachs
Alex Lowe
Ed Morley
Lewis Gaul
Victorien
Eric Mark Martin
1hakusai1
KATO So
梦飞翔
Ankit Saini
Choudhry Abdullah
piegames
Damy Metzke
Seven4ME
ChiLin Chiu
Michał Górny
Andrew Gallant
Chao Ning
Mathieu Kniewallner
Scott Sanderson
Alex Lowe
Geoffrey Thomas
Matthieu Ancellin
Charles Tapley Hoyt
JackDyre