Microsoft Store Pythons do not always register themselves in the
registry, so we port
<58ce131037/PC/launcher2.c (L1744)>
and look them up on the filesystem in known locations.
## Test Plan
So far I've confirmed that we find a store Python when I use `cargo run
python list`, can we make this a part of any of the platform tests
maybe?
Our current strategy of parsing the output of `py --list-paths` to get
the installed python versions on windows is brittle (#6524, missing
`py`, etc.) and it's slow (10ms last time i measured).
Instead, we should behave spec-compliant and read the python versions
from the registry following PEP 514.
It's not fully clear which errors we should ignore and which ones we
need to raise.
We're using the official rust-for-windows crates for accessing the
registry.
Fixes#1521Fixes#6524
## Summary
This PR revives https://github.com/astral-sh/uv/pull/4944, which I think
was a good start towards adding `--trusted-host`. Last night, I tried to
add `--trusted-host` with a custom verifier, but we had to vendor a lot
of `reqwest` code and I eventually hit some private APIs. I'm not
confident that I can implement it correctly with that mechanism, and
since this is security, correctness is the priority.
So, instead, we now use two clients and multiplex between them.
Closes https://github.com/astral-sh/uv/issues/1339.
## Test Plan
Created self-signed certificate, and ran `python3 -m http.server --bind
127.0.0.1 4443 --directory . --certfile cert.pem --keyfile key.pem` from
the packse index directory.
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
https://127.0.0.1:8443/simple-html` failed with:
```
error: Request failed after 3 retries
Caused by: error sending request for url (https://127.0.0.1:8443/simple-html/transitive-yanked-and-unyanked-dependency-a-0abad3b6/)
Caused by: client error (Connect)
Caused by: invalid peer certificate: Other(OtherError(CaUsedAsEndEntity))
```
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.1:8443'`
failed with the expected error (invalid resolution) and made valid
requests.
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.2' -n` also
failed.
As described in #4242, we're currently incorrectly downloading glibc
python-build-standalone on musl target, but we also can't fix this by
using musl python-build-standalone on musl targets since the musl builds
are effectively broken.
We reintroduce the libc detection previously removed in #2381, using it
to detect which libc is the current one before we have a python
interpreter. I changed the strategy a big to support an empty `PATH`
which we use in the tests.
For simplicity, i've decided to just filter out the musl
python-build-standalone archives from the list of available archive,
given this is temporary. This means we show the same error message as if
we don't have a build for the platform. We could also add a dedicated
error message for musl.
Fixes#4242
## Test Plan
Tested manually.
On my ubuntu host, python downloads continue to pass:
```
target/x86_64-unknown-linux-musl/debug/uv python install
```
On alpine, we fail:
```
$ docker run -it --rm -v .:/io alpine /io/target/x86_64-unknown-linux-musl/debug/uv python install
Searching for Python installations
error: No download found for request: cpython-any-linux-x86_64-musl
```
## Summary
This is similar to https://github.com/astral-sh/uv/pull/6171 but more
expansive... _Anywhere_ that we test requirements for platform
compatibility, we _need_ to respect the resolver-friendly markers. In
fixing the motivating issue (#6621), I also realized that we had a bunch
of bugs here around `pip install` with `--python-platform` and
`--python-version`, because we always performed our `satisfy` and `Plan`
operations on the interpreter's markers, not the adjusted markers!
Closes https://github.com/astral-sh/uv/issues/6621.
For various reasons, I have a preference for out of tree virtual
environments. Things just work if I symlink, but I don't know that this
is guaranteed, so I thought I'd add a test for it. It looks like there's
another code path that matters (`FoundInterpreter::discover ->
PythonEnvironment::from_root`) for the higher level commands, but
couldn't spot a good place to test that.
Related discussion:
https://github.com/astral-sh/uv/issues/1495#issuecomment-1950442191 /
https://github.com/astral-sh/uv/issues/1578#issuecomment-1949911871
In https://github.com/astral-sh/uv/pull/6359, I accidentally made `uv
python install` prefer `.python-version` files over `.python-versions`
files -.-, kind of niche but it's a regression.
## Summary
resolves https://github.com/astral-sh/uv/issues/5915, not entirely sure
if `manylinux_compatible` should be a separate field in the JSON
returned by the interpreter or there's some way to use the existing
`platform` for it.
## Test Plan
ran the below
```
rm -rf .venv
target/debug/uv venv
# commenting out the line below triggers the change..
# target/debug/uv pip install no-manylinux
target/debug/uv pip install cryptography --no-cache
```
is there an easy way to add this into the existing snapshot-based test
suite? looking around to see if there's a way that doesn't involve
something implementation-dependent like mocks.
~update: i think the output does differ between these two, so probably
we can use that.~ i lied - that "building..." output seems to be
discarded.
Executable name requests were being treated as explicit requests to
install into system environments, but I don't think it should be as it's
implicit what environment you'll end up in. Following #4308, we allow
multiple executables to be found so we can filter here.
Concretely, this means `--system` is required to install into a system
environment discovered with e.g. `--python=python`. The flag is still
not required for cases where we're not mutating environment.
- Removes "experimental" labels from command documentation
- Removes preview warnings
- Removes `PreviewMode` from most structs and methods — we could keep it
around but I figure we can propagate it again easily where needed in the
future
- Enables preview behavior by default everywhere, e.g., `uv venv` will
download Python versions
The loose consensus is that "fetch" doesn't have much meaning and that a
boolean flag makes more sense from the command line.
1. Adds `--allow-python-downloads` (hidden, default) and
`--no-python-downloads` to the CLI to quickly enable or disable
downloads
2. Deprecates `--python-fetch` in favor of the options from (1)
3. Removes `python-fetch` in favor of a `python-downloads` setting
5. Adds a `never` variant to the enum, allowing even explicit installs
to be disabled via the configuration file
## Test plan
I tested this with various `pyproject.toml`-level settings and `uv venv
--preview --python 3.12.2` and `uv python install 3.12.2` with and
without the new CLI flags.
## Summary
This PR adds a `DistExtension` field to some of our distribution types,
which requires that we validate that the file type is known and
supported when parsing (rather than when attempting to unzip). It
removes a bunch of extension parsing from the code too, in favor of
doing it once upfront.
Closes https://github.com/astral-sh/uv/issues/5858.
## Summary
It's fine for this to be in the cache, I think, since we don't
necessarily need to colocate it with the Python directory.
Closes https://github.com/astral-sh/uv/issues/5747.
Collapses the previous default into "managed" and makes the "managed"
behavior match "installed". People should use "only-managed" if they
want that behavior, it seems overly complicated otherwise.
## Summary
This PR deprecates the `--isolated` flag. The treatment varies across
the APIs:
- For non-preview APIs, we warn but treat it as equivalent to
`--no-config`.
- For preview APIs, we warn and ignore it, with two exceptions...
- For `tool run` and `run` specifically, we don't even warn, because we
can't differentiate the command-specific `--isolated` from the global
`--isolated`.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
use windows-sys bindings maintained by microsoft devs. winapi didn't has
any updates for more than 3 years
## Test Plan
cargo test. it failed locally because I don't have Python 3.12 installed
## Summary
uv run --directory <path> means that one doesn't have to change to a
project's directory to run programs from it. It makes it possible to use
projects as if they are tool installations.
To support this, first the code reading .python-version was updated so
that
it can read such markers outside the current directory. Note the minor
change this causes (if I'm right), described in the commit.
## Test Plan
One test has been added.
## --directory
Not sure what the name of the argument should be, but it's following uv
sync's directory for now.
Other alternatives could be "--project". Uv run and uv tool run should
probably find common agreement on this (relevant for project-locked
tools).
I've implemented this same change in Rye, some time ago, and then we
went
with --pyproject `<`path to pyproject.toml file`>`. I think using
pyproject.toml file path and not directory was probably a mistake, an
overgeneralization one doesn't need.
## Summary
Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:
* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
`.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.
The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).
Fixes: #3863
## Test Plan
* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
> DEBUG Found `cpython-3.12.1-macos-aarch64-none` at
`/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
Instead of `<implementation> <version>`
> DEBUG Found cpython 3.12.1 at `/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
## Summary
Similiar to https://github.com/astral-sh/rye/pull/680, I have made a
major refactor to the `fetch-download-metadata.py` script.
Some notable changes:
- Use PEP 723 inline scripts
- Fully type annotated the script
- Implemented async HTTP fetching
- Introduced a `Finder` base class and move finder logic under
`CPythonFinder` subclass, which will make it easier to add a
`PyPyFinder` later.
- Instead of fetching `xxx.sha256` for each file, the script now fetches
a single `SHA256SUMS` file containing checksums for all files in the
release.
As a result, the script now takes around 10 seconds instead of 10+
minutes.
## Plan for Future PRs
- [ ] Implement the `PyPyFinder`
- [ ] Add an GitHub Action to run `fetch-download-metadata.py` daily and
create PR automatically
## Test Plan
```sh
cargo run -- run --isolated -- ./crates/uv-python/fetch-download-metadata.py
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Currently, `uv` refuses to install anything on GraalPy. This is
currently blocking GraalPy testing with cibuildwheel, since manylinux
includes both `uv` and `graalpy` (but doesn't test with `uv`), whereas
cibuildwheel defaults to `uv`. See e.g.
https://github.com/pypa/cibuildwheel/actions/runs/9956369360/job/27506182952?pr=1538
where it gives
```
+ python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
* Creating isolated environment: venv+uv...
* Using external uv from /usr/local/bin/uv
* Installing packages in isolated environment:
- setuptools >= 40.8.0
> /usr/local/bin/uv pip install "setuptools >= 40.8.0"
< error: Unknown implementation: `graalpy`
```
## Test Plan
I simply based the GraalPy support on PyPy and added some small tests.
I'm open to discussing how to test this. GraalPy is available for
manylinux images and with setup-python, so we should be able to add
tests against it to the CI. I locally confirmed by installing `uv` into
a GraalPy venv and then trying things like `uv pip install Pillow` and
testing those extensions.
Search for all `python3.x` minor versions in PATH, skipping those we
already know we can use.
For example, let's say `python` and `python3` are Python 3.10. When a
user requests `>= 3.11`, we still need to find a `python3.12` in PATH.
We do so with a regex matcher.
Fixes#4709
## Summary
Resolves#5139
`PythonInstallationKey` was sorted as a string, which caused `3.8` to
appear before `3.11`. This update changes the sorting of
`PythonInstallationKey` to be a descending order by version.
## Test Plan
```sh
$ cargo run -- python install 3.8 3.12
$ cargo run -- tool run -v python -V
DEBUG uv 0.2.25
warning: `uv tool run` is experimental and may change without warning.
DEBUG Searching for Python interpreter in managed installations, system path, or `py` launcher
DEBUG Searching for managed installations at `C:\Users\xx\AppData\Roaming\uv\data\python`
DEBUG Found managed Python `cpython-3.12.3-windows-x86_64-none`
DEBUG Found cpython 3.12.3 at `C:\Users\xx\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none\install\python.exe` (managed installations)
DEBUG Using request timeout of 30s
DEBUG Using request timeout of 30s
DEBUG Acquired lock for `C:\Users\nigel\AppData\Roaming\uv\data\tools`
DEBUG Using existing environment for tool `httpx`: C:\Users\xx\AppData\Roaming\uv\data\tools\httpx
DEBUG Using existing tool `httpx`
DEBUG Running `httpx -v`
```
Fix#4988
## Summary
Running `uv python list` on glibc-based Linux will list musl pythons.
```bash
$ uv version
uv 0.2.24
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.11.9-linux-x86_64-musl <download available>
cpython-3.10.14-linux-x86_64-musl <download available>
cpython-3.9.19-linux-x86_64-musl <download available>
cpython-3.8.19-linux-x86_64-musl <download available>
cpython-3.7.9-linux-x86_64-musl <download available>
```
Change it to show Python matching the environment's libc as follows.
```bash
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.11.9-linux-x86_64-gnu <download available>
cpython-3.10.14-linux-x86_64-gnu <download available>
cpython-3.9.19-linux-x86_64-gnu <download available>
cpython-3.8.19-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
```
Also, if --all-platforms is specified, change to list Python for all
architectures and libc.
```bash
$ uv python list --all-platforms
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-windows-x86_64-none <download available>
cpython-3.12.3-windows-x86-none <download available>
cpython-3.12.3-macos-x86_64-none <download available>
cpython-3.12.3-macos-aarch64-none <download available>
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.12.3-linux-s390x-gnu <download available>
cpython-3.12.3-linux-powerpc64le-gnu <download available>
cpython-3.12.3-linux-armv7-gnueabihf <download available>
cpython-3.12.3-linux-armv7-gnueabi <download available>
cpython-3.12.3-linux-aarch64-gnu <download available>
...
```
## Test Plan
The following commands were executed on the command line to confirm the
results in Ubuntu 24.04.
- `cargo run python list`
- `cargo run python list --all-platforms`
## Summary
Closes https://github.com/astral-sh/uv/issues/4915.
```
error: No interpreter found for Python 3.12.4 in virtual environments, managed installations, or system path
```
## Summary
Closes https://github.com/astral-sh/uv/issues/4848.
## Test Plan
```
> cargo run -- run -vv --preview --isolated --python 3.12.4 python -V
error: No interpreter found for Python 3.12.4 in virtual environments or managed installations or system path
```
## Summary
Resolves#4834
## Test Plan
```sh
# 3.12.3 is a `install_only` archive
$ cargo run -- python install --preview --force 3.12.3
# 3.9.4 has only `full` archive
$ cargo run -- python install --preview --force 3.9.4
```
## Summary
Like https://github.com/astral-sh/uv/pull/4808 but with a few more
changes. I suspect this will require some bikeshedding but I find the
use of "installation" and "installed" in the same sentence to be kind of
a lot.
## Summary
Check the sha256 checksum when downloading a managed python toolchain.
## Test Plan
```sh
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12.3 (any-3.12.3-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
Installed Python 3.12.3 to C:\Users\jo\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none
Installed 1 installation in 6s
$ cargo run -- python uninstall 3.12
$ # manually change the hash in `crates/uv-python/src/downloads.inc`
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12 (any-3.12-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
error: Hash mismatch for `cpython-3.12.3-windows-x86_64-none`
Expected:
xx
Computed:
776568c92c5f3b47dbf5f17c1c58578f70d75a32654419a158aa8bdc6f95b09a
```
## Summary
The basic strategy:
- When the user does `uv tool run`, we resolve the `from` and `with`
requirements (always).
- After resolving, we generate a hash of the requirements. For now, I'm
just converting to a lockfile and hashing _that_, but that's an
implementation detail.
- Once we have a hash, we _also_ hash the interpreter.
- We then store environments in
`${CACHE_DIR}/${INTERPRETER_HASH}/${RESOLUTION_HASH}`.
Some consequences:
- We cache based on the interpreter, so if you request a different
Python, we'll create a new environment (even if they're compatible).
This has the nice side-effect of ensuring that we don't use environments
for interpreters that were later deleted.
- We cache the `from` and `with` together. In practice, we may want to
cache them separately, then layer them? But this is also an
implementation detail that we could change later.
- Because we use the lockfile as the cache key, we will invalidate the
cache when the format changes. That seems ok, but we could improve it in
the future by generating a stable hash from a lockfile that's
independent of the schema.
Closes https://github.com/astral-sh/uv/issues/4752.
This fell out of my investigation of
https://github.com/astral-sh/uv/issues/4774 but the bug was fixed by the
reporter in #4775
- Adds support for `GH_TOKEN` authentication again — basically needed to
avoid rate limits when hacking on this.
- Clarifies some handling and logging of flavors
Fix#4774.
## Summary
Change the python interpreter for linux installed with `uv python` to an
optimized one.
## Test Plan
I ran the following command on Linux (glibc) to confirm that an
optimized (not debug built) Python is installed.
```bash
# install python
uv python install 3.12.3
# check build type
uv run python -c "import sysconfig;print(sysconfig.get_config_var('Py_DEBUG'))"
0
```
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
konsti
Zanie Blue
Charlie Marsh
github-actions[bot]
renovate[bot]
Shantanu
Chan Kang
Ben Beasley
Maksim Bondarenkov
bluss
Pavel Dikov
Jo
Tim Felgentreff
Di-Is