Initially, we were limiting Git schemes to HTTPS and SSH as only
supported schemes. We lost this validation in #3429. This incidentally
allowed file schemes, which apparently work with Git out of the box.
A caveat for this is that in tool.uv.sources, we parse the git field
always as URL. This caused a problem with #11425: repo = { git =
'c:\path\to\repo', rev = "xxxxx" } was parsed as a URL where c: is the
scheme, causing a bad error message down the line.
This PR:
* Puts Git URL validation back in place. It bans everything but HTTPS,
SSH, and file URLs. This could be a breaking change, if users were using
a git transport protocol were not aware of, even though never
intentionally supported.
* Allows file: URL in Git: This seems to be supported by Git and we were
supporting it albeit unintentionally, so it's reasonable to continue to
support it.
* It does not allow relative paths in the git field in tool.uv.sources.
Absolute file URLs are supported, whether we want relative file URLs for
Git too should be discussed separately.
Closes#3429: We reject the input with a proper error message, while
hinting the user towards file:. If there's still desire for relative
path support, we can keep it open.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Resolves#6913.
Add `tool.uv.build-constraint-dependencies` to pyproject.toml.
The changes are analogous to the constraint-dependencies feature
implemented in #5248.
Add documentation for `build-constraint-dependencies`
## Test Plan
Add tests for `uv lock`, `uv add`, `uv pip install` and `uv pip
compile`.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This typo wasn't caught because the `($arg:expr, false)` macro branch
was never exercised.
For example, prior to this change, if you add
```
show_settings!(globals, false);
```
below, you'll get a compiler error.
When running `uv pip install .` in a directory with a pyproject.toml
that does not configure a build, we will invoke setuptools and get a
wheel we can't parse (https://github.com/astral-sh/uv/issues/11344).
This PR adds warnings around these setups.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
## Summary
This PR fixes a subtle issue arising from our propagation of
preferences. When we resolve a fork, we take the solution from that fork
and mark all the chosen versions as "preferred" as we move on to the
next fork.
In this specific case, the resolver ended up solving a macOS-specific
fork first, which led us to pick `2.6.0` rather than `2.6.0+cpu`. This
in itself is correct; but when we moved on to the next fork, we
preferred `2.6.0` over `2.6.0+cpu`, despite the fact that `2.6.0` _only_
includes macOS wheel, and that branch was focused on Linux.
Now, in preferences, we prefer local variants (if they exist). If the
local variant ends up not working, we'll presumedly backtrack to the
base version anyway.
Closes https://github.com/astral-sh/uv/issues/11406.
## Summary
If the user provides a PEP 508 requirement (e.g., `uvx
change_wheel_version`), then we should us that verbatim for the
executable, rather than normalizing the package name.
Closes https://github.com/astral-sh/uv/issues/11521.
## Summary
This PR revives https://github.com/astral-sh/uv/pull/10017, which might
be viable now that we _don't_ enforce any platforms by default.
The basic idea here is that users can mark certain platforms as required
(empty, by default). When resolving, we ensure that the specified
platforms have wheel coverage, backtracking if not.
For example, to require that we include a version of PyTorch that
supports Intel macOS:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = ["torch>1.13"]
[tool.uv]
required-platforms = [
"sys_platform == 'darwin' and platform_machine == 'x86_64'"
]
```
Other than that, the forking is identical to past iterations of this PR.
This would give users a way to resolve the tail of issues in #9711, but
with manual opt-in to supporting specific platforms.
## Summary
This is an alternative to the approach we took in #11063 whereby we
always included `provides-extra` and `requires-dist`, since we needed
some way to differentiate between "no extras" and "lockfile was
generated by a uv version that didn't include extras".
Instead, this PR adds a minor version (called a "revision") to the
lockfile that we can use to indicate support for this feature. While
lockfile version bumps are backwards-incompatible, older uv versions
_can_ read lockfiles with a later revision -- they just won't understand
all the data.
In a future major version bump, we could simplify things and change the
schema to use a (major, minor) format instead of these two separate
fields. But this is the only way to do it that's backwards-compatible
with existing uv versions.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Initially it seemed like `app.py` might be slightly more desirable but
people seem to overwhelmingly favour `main.py` as a good "generic" name.
Fixes#7782
Closes#11285
Closes https://github.com/astral-sh/uv/pull/11437
This changes `-p` from an alias of `--python-version` to `--python`
while retaining backwards compatibility for `--python-version`-like
fallback behavior when the requested version, e.g., `-p 3.12`, cannot be
found.
This was initially implemented with a hidden `--python-legacy` flag
which allows us to special case the short `-p` flag — unlike the
implementation in #11437. However, after further discussion, we decided
the behavior difference between `-p` and `--python` would be confusing
so now `-p` is an alias for `--python` and `--python` is special-cased
when a version is used.
Additionally, we now respect the `UV_PYTHON` environment variable, but
it is ignored when `--python-version` is set. If you want different
`--python-version` and `--python` values, you must do so explicitly. I
considered banning this, but it is valid for e.g. `--python pypy
--python-version 3.12`
Unlike https://github.com/astral-sh/uv/pull/10222, this does not respect
`UV_PYTHON` in `uv python uninstall` (continuing to require an explicit
target there) which I think is simpler and matches our `.python-version`
file behavior.
---------
Co-authored-by: Choudhry Abdullah <cabdulla@trinity.edu>
Co-authored-by: Choudhry Abdullah <choudhry347@choudhrys-air-2.trinity.local>
Co-authored-by: Aria Desires <aria.desires@gmail.com>
Closes#10597.
Recreated https://github.com/astral-sh/uv/pull/10925 that got closed as
the base branch got merged.
Snapshot tests.
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
`uv publish` has not changed for some time, it has [notable production
usage](https://github.com/search?q=%22uv+publish%22&type=code) and there
are no outstanding blockers, it is time to stabilize it with the 0.6
release.
Publishing is only usable through `uv publish`. You need to build source
distributions and wheels ahead of time, usually with `uv build`.
By default, `uv publish` will upload all source distributions and wheels
in the `dist/` folder, ignoring all non-matching filenames. By default,
`uv build` and most other build frontend write their artifacts to
`dist/`. Together, we can build a publish workflow including a smoke
test that all relevant files have actually been included in the wheel:
```
uv build
uv venv
uv pip install --find-links dist ...
uv run smoke_test.py
uv publish
```
There are 3 options supported in configuration files:
- `tool.uv.publish-url`
- `tool.uv.trusted-publishing`
- `tool.uv.check-url`
Options support on the CLI and through environment variables for index
configuration:
```
--index <INDEX>
The name of an index in the configuration to use for publishing [env: UV_PUBLISH_INDEX=]
--publish-url <PUBLISH_URL>
The URL of the upload endpoint (not the index URL) [env: UV_PUBLISH_URL=]
--check-url <CHECK_URL>
Check an index URL for existing files to skip duplicate uploads [env: UV_PUBLISH_CHECK_URL=]
```
There are two ways to configure `uv publish`: Passing options
individually or using the index API.
For the individual options, there `--publish-url` and `--check-url`, and
their configuration counterparts, `tool.uv.publish_url` and
`tool.uv.check_url`. `--publish-url` is named this way to be clearly
different from the simple index URL, since uploading to the index URL
leads to unclear errors, or worse a 200 OK with no effect. While we
intend to keep supporting this configuration, the index API is better
integrated.
In the index API, the user specifies `[[tool.uv.index]]`, with an index
name, the simple index URL and the publish URL. The `publish-url` and
`url` are equivalent to `--publish-url` and `--check-url`. The `url`
being mandatory makes for a better upload behavior (next paragraph).
```toml
[[tool.uv.index]]
name = "pypi"
url = "https://pypi.org/simple"
publish-url = "https://upload.pypi.org/legacy/"
```
A version of a package contains multiple files, for pure-python packages
usually a source distribution and a wheel, for native packages usually
many, larger wheels and a source distributions. Uploads in the not
officially specified Upload API 1.0 are file based: Once you upload a
file, the version is created, even though most files are still missing.
When uploading a series of files fails in the middle (e.g. the CI server
breaks), the release is only half uploaded. For such cases, you want to
re-try the upload. The response of an index when re-uploading a file is
implementation defined. Notably, PyPI accepts uploads of the same file
again with status 200, but rejects uploads of a file with the same name
but different contents with status 400. Other indexes reject all
attempts at re-uploads with different status codes and messages. Twine
handles this with `--skip-existing`, which allows ignoring errors due to
files with the same name as an existing file being uploaded, however
this does also not error when uploading a file with different contents
but the same name, which indicates a problem with the publish pipeline.
To properly solve this, we need the ability to stage releases: Files of
a version are uploaded to a staging area, and only when all files are
uploaded, we atomically publish the release. When an upload breaks or CI
fails, we can discard or overwrite the staging area and try again. This
will only be properly solved by PEP 694 "Upload 2.0 API for Python
Package Indexes", with unclear progress. For local publishing, it would
also be convenient to be able to check which files exist and what their
hashes are from only the publish URL, so files in the `dist/` folder
from a previous release can be ignored.
In the Upload API 1.0, we need to upload transformed METADATA fields
along with the file as form-data. We currently upload only recognized
metadata fields, where we know how to translate the field name to the
form-data name. This means when a user adds unknown, wrong or future-PEP
metadata we miss it. To me best knowledge no index currently verifies
that the form-data and the METADATA file in the wheel match.
Upload API 2.0 will be an entirely new protocol. It is unclear how we
will decide whether to use Upload API 1.0 or Upload API 2.0 once the
latter is released. Upload API 2.0 will remove the need for a check URL.
This means no changes for `--index`, but `--check-url` will be
incompatible with Upload API 2.0.
Options support on the CLI and through environment variables for
authentication:
```
-u, --username <USERNAME>
The username for the upload [env: UV_PUBLISH_USERNAME=]
-p, --password <PASSWORD>
The password for the upload [env: UV_PUBLISH_PASSWORD=]
-t, --token <TOKEN>
The token for the upload [env: UV_PUBLISH_TOKEN=]
--trusted-publishing <TRUSTED_PUBLISHING>
Configure using trusted publishing through GitHub Actions [possible values: automatic, always,
never]
--keyring-provider <KEYRING_PROVIDER>
Attempt to use `keyring` for authentication for remote requirements files [env:
UV_KEYRING_PROVIDER=] [possible values: disabled, subprocess]
```
We need credentials for the publish URL, and we may need credentials for
the check URL.
We support credentials from environment variables, the CLI, the URL, the
keyring, trusted publishing or a prompt.
The username can come from, in order:
- Mutually exclusive:
- `--username` or `UV_PUBLISH_USERNAME`. The CLI option overrides the
environment variable
- The username field in the publish URL
- If `--token` or `UV_PUBLISH_TOKEN` are used, it is `__token__`. The
CLI option overrides the environment variable
- If trusted publishing is available, it is `__token__`
- (We currently do not read the username from the keyring)
- If stderr is a tty, prompt the user
The password can come from, in order:
- Mutually exclusive:
- `--password` or `UV_PUBLISH_PASSWORD`. The CLI option overrides the
environment variable
- The password field in the publish URL
- If `--token` or `UV_PUBLISH_TOKEN` are used, it is the token value.
The CLI option overrides the environment variable
- If the keyring is enabled, the keyring entry for the URL and username
- If trusted publishing is available, the trusted publishing token
- If stderr is a tty, prompt the user
If no credentials are found, we do a final check in the auth middleware
cache and otherwise error without sending the request.
Trusted publishing is only supported in GitHub Actions. By default, we
try to retrieve a token from it in GitHub Actions (`GITHUB_ACTIONS` is
`true`) but continue even it this fails. Trusted publishing can be
forced with `--trusted-publishing always`, to error on misconfiguration,
or deactivated with `--trusted-publishing never`. The option can also be
configured through `tool.uv.trusted-publishing`.
When `--check-url` or `--index` are used, we may need credentials for
the index URL, too. These are handle separately by the same rules as
using the index anywhere else. The `--keyring-provier` option is however
shared between them, turning the keyring on for either turns it on for
both.
As future option, we could read `UV_INDEX_USERNAME` and
`UV_INDEX_PASSWORD` as fallbacks for the publish credentials
(https://github.com/astral-sh/uv/issues/9845). This however would clash
with prompting: When index credentials and upload credentials are not
the same (they usually should be different, since regular uv operations
should have less privileges than publish), we would then instead of
prompting use the wrong credentials from `UV_INDEX_*` and fail.
A major UX problem is that there is no standard for the username when
using a token (or rather, there is no standard for just sending a token
without a username). PyPI uses `__token__`, Cloudsmith used to use your
username or `token`, but now also supports `__token__`
(https://github.com/astral-sh/uv/issues/8221), while Google Cloud
Artifacts always uses `oauth2accesstoken`
(https://github.com/astral-sh/uv/issues/9778). This means the index
documentation may say you're getting a token for authentication, but you
must not use `--token`, you must instead set username and password. This
is something that we can hopefully fix with Upload API 2.0.
An unsolved problem with the keyring is that you it's best practice to
use publish tokens scoped to projects and store tokens in a secure
location such as the keyring, but the keyring saves a single password
per publish URL and username combination. That means that it can't
natively store separate passwords for publishing multiple packages. The
current hack around this is using the package name as query parameter,
e.g. `https://test.pypi.org/legacy/?astral-test-keyring`, as PyPI
ignores this query parameter. This is however only applicable when
publishing locally and not from CI.
Another problem is that the keyring implementation currently relies on
the `keyring` pypi package, which needs to be installed in PATH together
with its plugins and is comparatively slow. This would be improved by
native keyring support (https://github.com/astral-sh/uv/issues/10867),
with the same caveats such as keyring plugins that shared with the
simple index API.
We currently don't upload attestations (PEP 740). Attestations are an
additional field in the form-data, so we should be able to add them
transparently without any changes to the API, unless we want to add a
switch to deactivate even when trusted publishing is used. See also
https://trailofbits.github.io/are-we-pep740-yet/.
Setuptools is writing an invalid combination of Metadata-Version and
used metadata fields in some cases, which PyPI correctly rejects
(https://github.com/astral-sh/uv/issues/9513).
We set a 15min overall timeout since reqwest is missing a write timeout
option (https://github.com/seanmonstar/reqwest/issues/2403).
https://github.com/astral-sh/uv/issues/8641 and
https://github.com/astral-sh/uv/issues/8774: We build artifact checking
in some capacity. This should be done ideally by the build backend or at
latest as part of `uv build`, doing it as part of publish is too late.
Closes#7839
---
Let me know if i missed anything.
We added this to help with resolving some specific packages, and for
parity with Poetry. But in some cases, this metadata is just wrong, and
at the very least it's unreliable.
Closes https://github.com/astral-sh/uv/issues/8989.
Closes#10945.
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.
Closes#11263.
We've never bumped the version of this bucket, and we may never do so...
But it's still incorrect for us to omit it from these serialized structs
in the cache. Specifically, these structs include a pointer into the
archive bucket (namely, the ID). But we don't include the bucket
version! So, in theory, we could end up pointing to archives that don't
match the current bucket version expected in the code.
## Summary
Just a logic issue... If we see a dynamic field that isn't `"version"`,
we end up _not_ propagating the fact that `"version"` is also dynamic.
Closes https://github.com/astral-sh/uv/issues/11460.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Handle potential infinite recursion if `uv run` recursively invokes `uv
run`. This can happen if the shebang line of a script includes `uv run`,
but does not pass `--script`.
Handled by adding a new environment variable `UV_RUN_RECURSION_DEPTH`,
which contains a counter of the number of times that uv run has been
recursively invoked. If unset, it defaults to zero, and each time uv run
starts a subprocess we increment the counter, erroring if the value is
greater than a configurable (but not currently exposed or documented)
threshold.
Closes https://github.com/astral-sh/uv/issues/11220.
## Test Plan
I've added a snapshot test to `uv/crates/uv/tests/it/run` that tests the
end-to-end recursion detection flow. I've currently made it a unix-only
test because I'm not sure offhand how uv run will interact with shebang
lines on windows.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Closes https://github.com/astral-sh/uv/issues/11288
I tested the reproduction there manually.
I'm a little uncertain about this behavior, it's not true to the spirit
of `--python <dir>` selecting a target environment but this method is
only used to see if an existing environment matches for the purpose of
invalidation in projects and tools where I think we always force a
separate environment anyway?
## Summary
We need to add indexes in the order in which they're respected by the
resolver. Otherwise, we risk writing an index to the `pyproject.toml`
that is canonically equal (but not verbatim equivalent) to the index we
use during resolutin.
Closes https://github.com/astral-sh/uv/issues/11312.
## Summary
The environment is located at a stable path within the cache, based on
the script's absolute path.
If a lockfile exists for the script, then we use our standard lockfile
semantics (i.e., update the lockfile if necessary, etc.); if not, we
just do a `uv pip sync` (roughly).
Example usage:
```
❯ uv init --script hello.py
Initialized script at `hello.py`
❯ uv add --script hello.py requests
Updated `hello.py`
❯ cargo run sync --script hello.py
Using script environment at: /Users/crmarsh/.cache/uv/environments-v1/hello-84e289fe3f6241a0
Resolved 5 packages in 3ms
Installed 5 packages in 12ms
+ certifi==2025.1.31
+ charset-normalizer==3.4.1
+ idna==3.10
+ requests==2.32.3
+ urllib3==2.3.0
```
Closes https://github.com/astral-sh/uv/issues/6637.
## Summary
This is a follow-on to #11347 to use a stable directory for remote and
stdin scripts. The annoying piece here was figuring out what to use as
the cache key. For remote scripts, I'm using the URL; for stdin scripts,
there isn't any identifying information, so I'm just using a hash of the
metadata.
## Summary
Today, scripts use `CachedEnvironment`, which results in a different
virtual environment path every time the interpreter changes _or_ the
project requirements change. This makes it impossible to provide users
with a stable path to the script that they can use for (e.g.) directing
their editor.
This PR modifies `uv run` to use a stable path for local scripts (we
continue to use `CachedEnvironment` for remote scripts and scripts from
`stdin`). The logic now looks a lot more like it does for projects: we
`get_or_init` an environment, etc.
For now, the path to the script is like:
`environments-v1/4485801245a4732f`, where `4485801245a4732f` is a SHA of
the absolute path to the script. But I'm not picky on that :)
## Summary
This PR refactors the whole `Target` abstraction, mostly to remove all
the repeated `From*` variants and logic in favor of a higher-level
struct that captures those details separately.
In doing so, it also adds support to `uvx` for unnamed requirements, for
parity with `uv tool install`. So, e.g., the following will show the
`flask` version:
```
uvx git+https://github.com/pallets/flask --version
```
I think this makes sense conceptually since we already support arbitrary
named requirements there.
## Summary
Right now, `uv sync --dry-run` returns the interpreter that _would've_
been used to create the environment; so we end up using _that_
interpreter's `site-packages`, and return changes relative to that
interpreter. Instead, we now create a temporary virtual environment and
compare against that.
Closes https://github.com/astral-sh/uv/issues/11422.
## Summary
We currently enforce that if you do `uv pip install
./dist/iniconfig-1.0.0.tar.gz`, the build _must_ produce a wheel like
`iniconfig-1.0.0-py3-none-any.whl` (i.e., the name and version must
match). It turns out some packages produce a wheel that has a local
suffix on it, like `vllm`. This PR makes the check a little more
permissive in that we now accept `1.0.0` or that version with a local
suffix (e.g., `1.0.0+cpu`). I don't love this practice, but we already
relaxed this check when _installing_ a wheel, so this seems reasonable:
5e15881dcc/crates/uv-install-wheel/src/install.rs (L50-L52)
Note that this is _still_ stricter than pip. pip seems to only require
that the package name is the same (i.e., `iniconfig` matches
`iniconfig`; but they'll happily install a wheel like
`iniconfig-2.0.0-py3-none-any.whl` given
`./dist/iniconfig-1.0.0.tar.gz`).
Closes https://github.com/astral-sh/uv/issues/11038.
A user reported a homebrew Python that would raise an exception in the
interpreter probing script because `platform.mac_ver()` returned `('',
('', '', ''), '')` on his installation due to
https://github.com/Homebrew/homebrew-core/issues/206778
This is easy enough to catch and show a proper error message instead of
the Python backtrace.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
This adds `NO_BINARY` and `NO_BINARY_PACKAGE` environment variables to
the uv CLI, allowing the user to specify packages to build from source
using environment variables. Its not a complete fix for #4291 as it does
not handle the `pip` subcommand.
## Test Plan
This was tested by running `uv sync` with various `UV_NO_BINARY` and
`UV_NO_BINARY_PACKAGE` environment variables set and checking that the
correct set of packages were compiled rather than taken from pre-built
wheels.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Now that `version` is an optional field, we shouldn't error if an
unambiguous package is lacking a version. We can still enforce the same
guarantees via `source`, since we always set version and source
together, if the package is unambiguous. I also retained the same error
for non-local packages that lack a version like this.
Closes https://github.com/astral-sh/uv/issues/11384.
The underlying cause here, I believe, was that we weren't accounting
for the case where an edge could be visited *without* any extras
enabled. Because of that, we got into situations where we thought
there was only one path to an edge when there were actually more
paths. This in turn lead to us erroneously doing simplification where
it actually isn't justified. And in turn lead to duplicate versions
of the same package being installed in the same environment.
The fix for this ends up being really simple: in the case where we
don't add any conflict items for a package during graph traversal,
we materialize an empty set of conflicts to mark the case of no
extras being enabled when visiting the child edges. This is enough
to propagate the knowledge of multiple paths to the same edge and
causes us to avoid doing improper simplifications.
This does fix the problem in the snapshot, but it does also I think
lead to other cases where simplifications are no longer possible
(hence the changes to the airflow snapshot). But this seems
expected, since we are doing strictly less simplification than we
were before. It's unclear if all of those cases were actual bugs
or not though.
The snapshot is too big to meaningfully read, but the problem is
in the dependencies of `torchmetrics`:
[[package]]
name = "torchmetrics"
version = "1.6.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "lightning-utilities" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "torch", version = "2.2.1", source = { registry = "https://pypi.org/simple" } },
{ name = "torch", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-4-test-chgnet' or extra != 'extra-4-test-m3gnet'" },
]
The conflict markers here are overlapping, which means
both can be included in the same environment.
Previously, we patched pkg-config .pc files to have the absolute path to
the directory where we unpack a python-build-standalone release. As
discussed in #11028, we can use ${pcfiledir} in a .pc file to indicate
paths relative to the location of the file itself.
This change was implemented in astral-sh/python-build-standalone#507, so
for newer python-build-standalone releases, we don't need to do any
patching. Optimize this case by only modifying the .pc file if an actual
change is needed (which might be helpful down the line with hard links
or something). For older releases, change uv's patch to match what
python-build-standalone now does.
## Motivation
No-op `uv lock` in apache airflow
(891c67f210ab7c877d1f00ea6ea3d3cdbb0e96ef) is slow, which makes `uv run`
slow, too.
Reference project:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 16.3 ms ± 1.5 ms [User: 9.8 ms, System: 6.4 ms]
Range (min … max): 13.0 ms … 20.0 ms 186 runs
```
Apache airflow before:
```
$ hyperfine "uv run python -c \"print('hi')\""
Benchmark 1: uv run python -c "print('hi')"
Time (mean ± σ): 161.0 ms ± 5.2 ms [User: 135.3 ms, System: 24.1 ms]
Range (min … max): 155.0 ms … 176.3 ms 18 runs
```
## Optimization
`FlatRequiresDist::from_requirements` is taking 50% of main thread
runtime.
Before:
After both commits:
Apache airflow after the first commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 122.3 ms ± 5.4 ms [User: 96.1 ms, System: 24.7 ms]
Range (min … max): 114.0 ms … 133.2 ms 23 runs
```
Apache airflow after the second commit:
```
$ hyperfine "uv-profiling run python -c \"print('hi')\""
Benchmark 1: uv-profiling run python -c "print('hi')"
Time (mean ± σ): 108.5 ms ± 3.4 ms [User: 83.2 ms, System: 24.2 ms]
Range (min … max): 103.6 ms … 119.9 ms 28 runs
```
## Summary
These are used for coordination across processes. If you run uv under,
e.g., the root user, then under a different user, I don't think we
should prevent you from acquiring the lock.
Closes https://github.com/astral-sh/uv/issues/11324.
Includes https://pypy.org/posts/2025/02/pypy-v7318-release.html
These are labeled as betas in the post but not anywhere obvious to me?
I'm not sure we need to portray this to users.
Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
## Summary
No behavior changes... This just separates the formatting from the
collection of the results, and also fixes a bug whereby we didn't say
"No changes detected" in some cases.
Given an input in the shape:
```
foo[bar]==1.0.0; sys_platform == 'linux'
foo==1.0.0; sys_platform != 'linux'
```
We would write either
```
foo==1.0.0; sys_platform == 'linux'
```
or
```
foo==1.0.0
```
depending on the iteration order, as the first one is from the marker
proxy package and the second one from the package without marker.
The fix correctly merges graph entries when there are two nodes with
different extras and different markers.
I tried to write a packse test but it failed due to a different
iteration order showing the correct case directly instead of the failing
one we'd need.
Only `strip_extras` is affected, since `combine_extras` uses
`version_marker`.
When stderr is not a tty, we currently don't show any messages for build
or large downloads, since indicatif is hidden. We can improve this by
showing a message for:
* Starting and finishing a large download (>1MB)
* Starting and finishing a build
Downloads are limited to 1MB or unknown size to keep the logs concise
and not scroll the entire terminal away for a download that finishes
almost immediately.
These messages are not captured in the tests since their order is
non-deterministic (downloads and builds race to finish).
There are no "tick" messages for large downloads yet, we could e.g. show
an update on runnning downloads every n seconds.
Part of #11121
**Test Plan**
```
$ uv venv && FORCE_COLOR=1 cargo run -q pip install numpy --no-binary :all: --no-cache 2>&1 | tee a.txt
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 1 package in 221ms
Building numpy==2.2.2
Built numpy==2.2.2
Prepared 1 package in 2m 34s
Installed 1 package in 6ms
+ numpy==2.2.2
```
```
$ uv venv && FORCE_COLOR=1 cargo run -q pip install torch --no-cache 2>&1 | tee b.txt
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 24 packages in 648ms
Downloading setuptools (1.2MiB)
Downloading nvidia-cuda-cupti-cu12 (13.2MiB)
Downloading torch (731.1MiB)
Downloading nvidia-nvjitlink-cu12 (20.1MiB)
Downloading nvidia-cufft-cu12 (201.7MiB)
Downloading nvidia-cuda-nvrtc-cu12 (23.5MiB)
Downloading nvidia-curand-cu12 (53.7MiB)
Downloading nvidia-nccl-cu12 (179.9MiB)
Downloading nvidia-cudnn-cu12 (634.0MiB)
Downloading nvidia-cublas-cu12 (346.6MiB)
Downloading sympy (5.9MiB)
Downloading nvidia-cusparse-cu12 (197.8MiB)
Downloading nvidia-cusparselt-cu12 (143.1MiB)
Downloading networkx (1.6MiB)
Downloading nvidia-cusolver-cu12 (122.0MiB)
Downloading triton (241.4MiB)
Downloaded setuptools
Downloaded networkx
Downloaded sympy
Downloaded nvidia-cuda-cupti-cu12
Downloaded nvidia-nvjitlink-cu12
Downloaded nvidia-cuda-nvrtc-cu12
Downloaded nvidia-curand-cu12
[...]
```
## Summary
This is attempting to solve the same problem surfaced in #11208 and
#11209. However, those PRs only worked for our own managed Pythons. In
Gentoo, for example, they disable the managed Pythons, which led to
failures in the test suite, because the "base Python" returned after
creating a virtual environment would differ from the "base Python" that
you get after _querying_ an existing virtual environment.
The fix here is to apply our same base Python normalization and
discovery logic, to non-standalone / non-managed Pythons. We continue to
use `sys._base_executable` for such Pythons when creating the
virtualenv, but when _caching_, we perform this second discovery step.
Closes https://github.com/astral-sh/uv/issues/11237.
This is a rewrite of the groups subsystem to have more clear semantics,
and some adjustments to the CLI flag constraints. In doing so, the
following bugs are fixed:
* `--no-default-groups --no-group foo` is no longer needlessly rejected
* `--all-groups --no-default-groups` now correctly evaluates to
`--all-groups` where previously it was erroneously being interpretted as
just `--no-default-groups`
* `--all-groups --only-dev` is now illegal, where previously it was
accepted and mishandled, as if it was a mythical `--only-all-groups`
flag
Fixes#10890Closes#10891
## Summary
If you `uv run` from the same directory via multiple processes at the
same time, some of them will fail as they'll see an "incomplete" virtual
environment.
Closes https://github.com/astral-sh/uv/issues/11219.
I think `UV_PROJECT_ENVIRONMENT` is too complicated for use-cases where
the user wants to sync to the active environment. I don't see a
compelling reason not to make opt-in easier. I see a lot of questions
about how to deal with this warning in the issue tracker, but it seems
painful to collect them here for posterity.
A notable behavior here — we'll treat this as equivalent to
`UV_PROJECT_ENVIRONMENT` so... if you point us to a valid virtual
environment that needs to be recreated for some reason (e.g., new Python
version request), we'll happily delete it and start over.
## Summary
This PR removes the ephemeral `.pth` overlay when using a cached
environment. This solution isn't _completely_ safe, since we could
remove the `.pth` file just as another process is starting the
environment... But that risk already exists today, since we could
_overwrite_ the `.pth` file just as another process is starting the
environment, so I think what I've added here is a strict improvement.
Ideally, we wouldn't write this file at all, and we'd instead somehow
(e.g.) pass a file to the interpreter to run at startup? Or find some
other solution that doesn't require poisoning the cache like this.
Closes https://github.com/astral-sh/uv/issues/11117.
# Test Plan
Ran through the great reproduction steps from the linked issue.
Before:
After:
## Summary
I'm not sure that this has much of an effect in practice, but currently,
when we return a virtual environment, the `sys_base_executable ` of the
parent ends up being retained as `sys_base_executable` of the created
environment. But these can be, like, subtly different? If you have a
symlink to a Python, then for the symlink, `sys_base_executable` will be
equal to `sys_executable`. But when you create a virtual environment for
that interpreter, we'll set `home` to the resolved symlink, and so
`sys_base_executable` will be the resolved symlink too, in general.
Anyway, this means that we should now have a consistent value between
(1) returning `Virtualenv` from the creation routine and (2) querying
the created interpreter.
## Summary
It turns out that we were returning slightly different interpreter paths
on repeated `uv run --with` commands. This likely didn't affect many (or
any?) users, but it does affect our test suite, since in the test suite,
we use a symlinked interpreter.
The issue is that on first invocation, we create the virtual
environment, and that returns the path to the `python` executable in the
environment. On second invocation, we return the `python3` executable,
since that gets priority during discovery. This on its own is
potentially ok. The issue is that these resolve to different
`sys._base_executable` values in these flows... The latter gets the
correct value (since it's read from the `home` key), but the former gets
the incorrect value (since it's just the `base_executable` of the
executable that created the virtualenv, which is the symlink).
We now use the same logic to determine the "cached interpreter" as in
virtual environment creation, to ensure consistency between those paths.
Closes https://github.com/astral-sh/uv/issues/11214
Special-cases the first Python executable we find on the `PATH`,
allowing it to be considered during searches for virtual environments.
For some context, there are two stages to Python interpreter discovery
1. We find possible Python executables in various sources
2. We query the executables to determine canonical metadata about the
interpreter
We can't really be "sure" if an executable is a complaint virtual
environment during (1), we need to query the interpreter first. This
means that if you're only allowed to installed into virtual
environments, we'll query every interpreter on your PATH. This is not
performant, and causes confusion for users. Notably, I recently improved
error messaging when we can't find any valid interpreters, by showing
the error message we encounter while querying an interpreter (if any).
However, this is problematic when there's an error for an interpreter
that is not relevant to your search. In
https://github.com/astral-sh/uv/pull/11143, I added filtering to avoid
querying additional interpreters, but that regressed some user
experiences where they were relying on us finding implicitly active
virtual environments via the PATH.
In https://github.com/astral-sh/tokio-tar/pull/2, we accidentally
changed the `target_base` from the target base to the parent of the
file. This would cause hardlink unpacking to fail.
Example: A hardlink at `hardlinked-0.1.0/pyproject.toml` pointing to
`hardlinked-0.1.0/pyproject.toml.real` would try pointing to
`hardlinked-0.1.0/hardlinked-0.1.0/pyproject.toml.real` instead and fail
the unpacking.
The actual fix is in astral-tokio-tar, on the uv side there are only tests.
Fixes#11213
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
I got a bit confused when testing `[dependency-groups]` because uv's
error message had the same typo I did in my `pyproject.toml`.
I tried to fix it, as well as a few comment I found along the way.
These are noisy relative to the effect they have on the user. It seems
better to prioritize hints on poor resolutions. Notably, it seems hard
to make these "not noisy" ref #11091.
Does not include the "lowest" resolution mode, in which lower bounds are
critical.
With the parallel simple index fetching, we would only acquire one
download concurrency token, meaning that we could in the worst case make
times the number of indexes more requests than the user requested limit.
We fix this by passing the semaphore down to the simple API method.
Looks like the set based prioritize tracking from
https://github.com/pubgrub-rs/pubgrub/pull/313 is a slight speedup.
I assume the changed derivation tree in the error snapshot is due to
out-of-sync virtual package priorities, while the main package priority
defining the solution remains stable.
```
$ hyperfine --warmup 2 "./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal" "./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal"
Benchmark 1: ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 115.0 ms ± 4.8 ms [User: 131.0 ms, System: 113.6 ms]
Range (min … max): 108.1 ms … 125.8 ms 25 runs
Benchmark 2: ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 105.4 ms ± 2.6 ms [User: 118.5 ms, System: 113.5 ms]
Range (min … max): 101.1 ms … 111.9 ms 28 runs
Summary
./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal ran
1.09 ± 0.05 times faster than ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
```
uv-install-wheel had the logic for laying out the installation and for
linking a directory in the same module. We split them up to isolate each
module's logic and tighten the crate's interface to only expose top
level members.
No logic changes, only moving code around.
As before, these are fine-grained PATs and will expire in 366 days.
They're generated by splitting the token into three parts (by `_`) and
base64 encoding.
## Summary
This lets us drop a dependency entirely. `percent-encoding` is used by
`url` and so is already in the graph, whereas `urlencoding` isn't used
by anything else.
## Summary
This PR adds an additional normalization step to `CanonicalUrl` whereby
we now percent-decode the path, to ensure that (e.g.)
`torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` and
`torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl` are considered
equal. Further, when generating the "reinstall" report, we use the
canonical URL rather than the verbatim URL.
In making this change, I also learned that we don't apply any of the
normalization passes to `file://` URLs. I inadvertently removed it in
93d606aba2,
since setting the password or URL on ` file://` URL errors -- but now
suppress those errors anyway.
Closes https://github.com/astral-sh/uv/issues/11082.
## Test Plan
- Downloaded a [PyTorch
wheel](https://download.pytorch.org/whl/cpu-cxx11-abi/torch-2.5.1%2Bcpu.cxx11.abi-cp39-cp39-linux_x86_64.whl)
- `python3.9 -m pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --platform
linux_x86_64 --target foo --no-deps`
- `cargo run pip install
torch-2.5.1+cpu.cxx11.abi-cp39-cp39-linux_x86_64.whl --python-platform
linux --python-version 3.9 --target foo --no-deps`
- Verified that the package had the `~` symbol for the reinstall.
## Summary
We now show a custom error if (1) the file doesn't exist at all, or (2)
it's not a PEP 723 script.
In the future, `uv lock --script` should probably initialize the script,
but that requires a more extensive refactor. At present, we just
silently lock the project instead, which is pretty bad!
Closes https://github.com/astral-sh/uv/issues/10979.
## Summary
I'm not a fan of registries including fragments here that aren't hashes,
but the spec doesn't expressly forbid it. I think it's reasonable to
ignore them.
Specifically, the spec is here:
https://packaging.python.org/en/latest/specifications/simple-repository-api/.
It says that:
> The URL **SHOULD** include a hash in the form of a URL fragment with
the following syntax: `#<hashname>=<hashvalue>`, where `<hashname>`he
lowercase name of the hash function (such as sha256) and `<hashvalue>`
is the hex encoded digest.
But it doesn't mention other fragments.
Closes https://github.com/astral-sh/uv/issues/7257.
## Summary
If we fail to deserialize cached metadata in the cache, we should just
ignore it, rather than failing.
Ideally, this never happens. If it does, it means we missed a cache
version bump. But if it does happen, it should still be non-fatal.
Closes https://github.com/astral-sh/uv/issues/11043.
Closes https://github.com/astral-sh/uv/issues/11101.
## Test Plan
Prior to this PR, the following would fail:
- `uvx uv@0.5.25 venv --python 3.12 --cache-dir foo`
- `uvx uv@0.5.25 pip install ./scripts/packages/hatchling_dynamic
--no-deps --python 3.12 --cache-dir foo`
- `uvx uv@0.5.18 venv --python 3.12 --cache-dir foo`
- `uvx uv@0.5.18 pip install ./scripts/packages/hatchling_dynamic
--no-deps --python 3.12 --cache-dir foo`
We can't go back and fix 0.5.18, but this will prevent such regressions
in the future.
Closes https://github.com/astral-sh/uv/issues/11048
This brings the `PythonEnvironment::from_root` behavior in-line with the
rest of uv Python discovery behavior (and in-line with pip). It's not
clear why we were canonicalizing the path in the first place here.
## Summary
This PR migrates all of our PyTorch tests to use our own mirror, which
includes upload timestamps that we can use to enforce
`--excludes-newer`, making the tests far more stable over time. (Today,
if you checkout old versions of `uv`, many of the PyTorch tests will
fail, since the index contents drift over time.)
Some snapshots changed in this PR (see, e.g.,
`universal_nested_overlapping_local_requirement`). The underlying reason
is that I used the current timestamp when setting upload times in the
PyTorch mirror, but those tests read from both the PyTorch
`--find-links` index _and_ PyPI. I guess we don't omit `--find-links`
entries based on `--excludes-newer`? That might be a bug. But I had to
_increase_ the `--excludes-newer` to include the PyTorch mirror's
`--find-links`, which meant pulling in some newer packages from PyPI
too. This is fine: it's a one-time churn, and they'll be stable going
forward.
In #10875, I relaxed the error checking during resolution to permit
dependencies like `foo[x1]`, where `x1` was defined to be conflicting.
In exchange, the error was, roughly speaking, moved to installation
time. This was achieved by looking at the full set of enabled extras
and checking whether any conflicts occurred. If so, an error was
reported. This ends up being more expressive and permits more valid
configurations.
However, in so doing, there was a bug in how the accumulated extras
were being passed to conflict marker evaluation. Namely, we weren't
accounting for the fact that if `foo[x1]` was enabled, then that fact
should be carried through to all conflict marker evaluations. This is
because some of those will use things like `extra != 'x1'` to indicate
that it should only be included if an extra *isn't* enabled.
In #10985, this manifested with PyTorch where `torch==2.4.1` and
`torch==2.4.1+cpu` were being installed simultaneously. Namely, the
choice to install `torch==2.4.1` was not taking into account that
the `cpu` extra has been enabled. If it did, then it's conflict
marker would evaluate to `false`. Since it didn't, and since
`torch==2.4.1+cpu` was also being included, we ended up installing both
versions.
The approach I took in this PR was to add a second breadth first
traversal (which comes first) over the dependency tree to accumulate all
of the activated extras. Then, only in the second traversal do we
actually build up the resolution graph.
Unfortunately, I have no automatic regression test to include here. The
regression test we _ought_ to include involves `torch`. And while we are
generally find to use those in tests that only generate a lock file, the
regression test here actually requires running installation. And
downloading and installing `torch` in tests is bad juju. So adding a
regression test for this is blocked on better infrastructure for PyTorch
tests. With that said, I did manually verify that the test case in #10985
no longer installs multiple versions of `torch`.
Fixes#10985
Charlie Marsh
konsti
Chao Ning
Zanie Blue
John Mumm
Jo
github-actions[bot]
Aria Desires
Mathieu Kniewallner
Scott Sanderson
Michał Górny
Alex Lowe
Andrew Gallant
renovate[bot]
Geoffrey Thomas
Matthieu Ancellin
Charles Tapley Hoyt
JackDyre
Sede Soukossi