Commit Graph

363 Commits

Author SHA1 Message Date
Geoffrey Thomas c8479574f2
Sync latest Python releases (#13593)
Pick up python-build-standalone 20250522, in particular to fix
astral-sh/python-build-standalone#619
2025-05-22 09:21:44 -04:00
konsti bc7b71f269
Platform discovery is using ld.so instead ldd (#13552)
In platform discovery we're parsing the output of the ELF interpreter,
e.g., `/lib64/ld-linux-x86-64.so.2`. This file is ld, not ldd, which was
incorrectly named in the code.

An alternative is naming everything ELF interpreter instead of ld.so.
2025-05-20 15:27:22 +02:00
Charlie Marsh c5032aee80
Bump MSRV to 1.85 and Edition 2024 (#13516)
## Summary

Builds on https://github.com/astral-sh/uv/pull/11724.

Closes https://github.com/astral-sh/uv/issues/13476.
2025-05-18 19:38:43 -04:00
Zanie Blue 4b7f5f1103
Update Python releases (#13509) 2025-05-17 13:19:20 -05:00
konsti 5d37c7ecc5
Apply first set of Rustfmt edition 2024 changes (#13478)
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.

Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
2025-05-16 20:19:02 -04:00
konsti 23261b7e2e
Apply some Edition 2024 fixes (#13479)
Some `ref`-removal and `use<>` fixes for the Rust Edition 2024
migration, which are also compatible with Rust Edition 2021.
2025-05-16 10:34:22 +02:00
Zanie Blue e92c17881c
Log skip reasons during Python installation key interpreter match checks (#13472)
Investigating https://github.com/astral-sh/uv/issues/13471
2025-05-15 10:28:01 -05:00
konsti 12860a9041
Remove extra newline (#13461)
This left an extra newline the hint, which isn't captured in the
snapshot.
2025-05-15 09:30:21 +00:00
konsti 3b125dbe71
Avoid panics for cannot-be-a-base URLs (#13406)
Following #13376, avoid `.unwrap()` on `Url::path_segments()`.

I also added some unwrap-safety comments.
2025-05-12 22:29:26 -04:00
konsti 1afadda819
Add more context to errors in high level crates. (#13351)
Add error context in some places where it was previously missing, and a
few style improvements.
2025-05-12 11:04:41 +00:00
Zanie Blue fa8db5a8d0
Avoid enumerating sources in errors for path Python requests (#13335)
e.g., these are misleading cruft in the error message at
https://github.com/astral-sh/uv/pull/12168#discussion_r2078204601

```
❯ uv python find /foo/bar
error: No interpreter found for path `/foo/bar` in virtual environments, managed installations, or search path
❯ cargo run -q -- python find /foo/bar
error: No interpreter found at path `/foo/bar`
```
2025-05-07 18:53:09 +00:00
konsti 364e3999d4
Improve error message when a virtual environment Python symlink is broken (#12168)
When removing a Python interpreter underneath an existing venv, uv
currently shows a not found error:

```
error: Failed to inspect Python interpreter from active virtual environment at `.venv/bin/python3`
  Caused by: Python interpreter not found at `/home/konsti/projects/uv/.venv/bin/python3`
```

This is unintuitive, as the file for the Python interpreter does exist,
it is a broken symlink that needs to be replaced with `uv venv`.

I've been encountering those occasionally, and I expect users that
switch between versions a lot will, too, especially when they also use
pyenv or a similar Python manager.

The new error hints at this solution:

```
error: Failed to inspect Python interpreter from active virtual environment at `.venv/bin/python3`
  Caused by: Broken symlink at `.venv/bin/python3`, was the underlying Python interpreter removed?

hint: To recreate the virtual environment, run `uv venv`
```
2025-05-07 13:24:53 -05:00
Zanie Blue ddd6f3debe
Fix discovery of pre-release managed Python versions in range requests (#13330)
We have test coverage for this elsewhere, but managed Python versions
are a distinct case because we know the _full_ version before querying
the interpreter (whereas, when we find them on the `PATH`, we usually
only know `X.y` from the file name).

This pre-filter logic now matches our subsequent logic at


060be9cef1/crates/uv-python/src/discovery.rs (L2146-L2149)



060be9cef1
shows the snapshot change.
2025-05-07 09:27:13 -05:00
Tim Felgentreff 878c2acdf3
Add downloading of GraalPy (#13172)
## Summary

This adds GraalPy download metadata so that `uv python install graalpy`
works. See https://github.com/astral-sh/uv/issues/13114

## Test Plan

The existing integration test was changed to test this functionality.
2025-05-06 11:02:27 -05:00
konsti 3218e364ae
Use `fs_err` for paths in symlinking errors (#13303)
In #13302, there was an IO error without context. This error seems to be
caused by a symlink error. Switching as symlinking to `fs_err` ensures
these errors will carry context in the future.
2025-05-05 16:29:27 +00:00
konsti b442aae80e
Only rebuild download JSON on change (#13261)
By default, Cargo runs the build script if any file in the package
changes
(https://doc.rust-lang.org/cargo/reference/build-scripts.html#change-detection).
In our case, we only need to rerun it if `download-metadata.json`
changed.
2025-05-02 07:49:34 -05:00
Meitar Reihan 0593b967ba
Add `python-downloads-json-url` option for `uv.toml` to configure custom Python installations via JSON URL (#12974)
## Summary

Part of #12838. Allow users to configure `python-downloads-json-url` in
`uv.toml` and not just from env.

I followed similar PR #8695, so same as there it's also available in the
CLI (I think maybe it's better not to be configurable from the CLI, but
since the mirror parameters are, I think it's better to do the same)


## Test Plan

<!-- How was it tested? -->
2025-04-30 15:52:11 -04:00
Meitar Reihan 5ee54b4fa3
minify and filter embed managed pythons json on compile time (#12967)
## Summary

In #10939 I added the generated
`crates/uv-python/src/download-metadata-minified.json` file which is a
minified version of `crates/uv-python/download-metadata.json`.

The main reason for this PR is to avoid bloating the git objects as this
is a single-line file.

As a bonus, I also filtered the embed json to include only the versions
for the compiled target. Which should improve the binary size and
performance by a bit.

## Test Plan

<!-- How was it tested? -->
2025-04-30 15:51:03 -04:00
Zanie Blue a9ab39ad6f
Fix patching of `clang` in managed Python sysconfig (#13237)
Regressed in
https://github.com/astral-sh/uv/pull/12239/files#r2069106892 because
additional entries override the previous one in the mapping. Now, we can
apply multiple patches in-order.

Closes #13236
2025-04-30 12:23:22 -05:00
Zanie Blue 3a87b6374a
Fix incorrect venv invalidation for pre-release Python versions (#13234)
I think this regressed in https://github.com/astral-sh/uv/pull/13027 — I
misunderstood what versions could be represented in the `pyvenv.cfg` (I
assumed they _never_ included pre-release components).

Closes #13233
2025-04-30 10:55:22 -05:00
Zanie Blue b6df755c9b Omit Python 3.7 downloads from managed versions (#13022)
Removes Python 3.7 installation support.

The following are available today and would no longer be available

```
cpython-3.7.9-windows-x86_64-none                      <download available>
cpython-3.7.9-windows-x86-none                         <download available>
cpython-3.7.9-macos-x86_64-none                        <download available>
cpython-3.7.9-linux-x86_64-gnu                         <download available>
cpython-3.7.7-windows-x86_64-none                      <download available>
cpython-3.7.7-windows-x86-none                         <download available>
cpython-3.7.7-macos-x86_64-none                        <download available>
cpython-3.7.7-linux-x86_64-gnu                         <download available>
cpython-3.7.6-windows-x86_64-none                      <download available>
cpython-3.7.6-windows-x86-none                         <download available>
pypy-3.7.13-windows-x86_64-none                        <download available>
pypy-3.7.13-macos-x86_64-none                          <download available>
pypy-3.7.13-linux-x86_64-gnu                           <download available>
pypy-3.7.13-linux-x86-gnu                              <download available>
pypy-3.7.13-linux-s390x-gnu                            <download available>
pypy-3.7.13-linux-aarch64-gnu                          <download available>
pypy-3.7.12-windows-x86_64-none                        <download available>
pypy-3.7.12-macos-x86_64-none                          <download available>
pypy-3.7.12-linux-x86_64-gnu                           <download available>
pypy-3.7.12-linux-x86-gnu                              <download available>
pypy-3.7.12-linux-s390x-gnu                            <download available>
pypy-3.7.12-linux-aarch64-gnu                          <download available>
pypy-3.7.10-windows-x86_64-none                        <download available>
pypy-3.7.10-macos-x86_64-none                          <download available>
pypy-3.7.10-linux-x86_64-gnu                           <download available>
pypy-3.7.10-linux-x86-gnu                              <download available>
pypy-3.7.10-linux-s390x-gnu                            <download available>
pypy-3.7.10-linux-aarch64-gnu                          <download available>
pypy-3.7.9-windows-x86-none                            <download available>
pypy-3.7.9-macos-x86_64-none                           <download available>
pypy-3.7.9-linux-x86_64-gnu                            <download available>
pypy-3.7.9-linux-x86-gnu                               <download available>
pypy-3.7.9-linux-s390x-gnu                             <download available>
pypy-3.7.9-linux-aarch64-gnu                           <download available>
```

All the CPython ones should absolutely not be available, as they're not
even the last security patch release. I'm on the fence about PyPy?
2025-04-29 16:37:00 -05:00
Zanie Blue 6cc2202799 Ignore arbitrary Python requests in version files (#12909)
Closes https://github.com/astral-sh/uv/issues/12605
2025-04-29 16:37:00 -05:00
Zanie Blue 1988e209ef Treat empty `UV_PYTHON_INSTALL_DIR` as unset (#12907)
Same as https://github.com/astral-sh/uv/pull/12905 — found via regex
2025-04-29 16:37:00 -05:00
konsti b33a19689c
Optional managed Python archive download cache (#12175)
Part of #11834

Currently, all Python installation are a streaming download-and-extract.
With this PR, we add the `UV_PYTHON_CACHE_DIR` variable. When set, the
installation is split into downloading the interpreter into
`UV_PYTHON_CACHE_DIR` and extracting it there from a second step. If the
archive is already present in `UV_PYTHON_CACHE_DIR`, we skip the
download.

The feature can be used to speed up tests and CI. Locally for me, `cargo
test -p uv -- python_install` goes from 43s to 7s (1,7s in release mode)
when setting `UV_PYTHON_CACHE_DIR`. It can also be used for offline
installation of Python interpreter, by copying the archives to a
directory in the offline machine, while the path rewriting is still
performed on the target machine on installation.
2025-04-28 12:09:09 +02:00
Zanie Blue fb08116800
Report Python versions in `pyvenv.cfg` version mismatch (#13027)
When working on #13025 I noticed this message was lacking versions,
which seems frustrating if you're debugging things.

I refactored the general `matches_interpreter` utilities that were added
in https://github.com/astral-sh/uv/pull/12884 into a more purpose-fit
function that returns an `Option` with the versions if there's a
mismatch.
2025-04-25 18:06:46 +00:00
Charlie Marsh a4ea814159
Avoid panic for invalid Python versions (#13077)
## Summary

We unwrap these further on, so we should validate them ahead of time.

Closes https://github.com/astral-sh/uv/issues/13075.
2025-04-23 22:23:33 +00:00
Zanie Blue 9484e3663c
Prefer stable releases over pre-releases in `uv python install` (#12194)
e.g., `uv python install 3` should not install the 3.14 alpha

Closes #12184
2025-04-21 21:16:07 +00:00
Jo 2cb998847c
Fix `SourceNotAllowed` error message (#13012)
## Summary

Before:

```console
$ uv python list py --managed-python
error: Interpreter discovery for `executable name `py`` requires `search path` but only only managed is allowed
```

After:

```console
$ uv python list py --managed-python
error: Interpreter discovery for `executable name `py`` requires `search path` but only `only managed` is allowed
```
2025-04-21 08:29:45 -04:00
Zanie Blue 41ac6649d7
Add debug logs for version file search (#12951)
Co-authored-by: John Mumm <jtfmumm@gmail.com>
2025-04-18 16:31:52 +00:00
Zanie Blue 91410acf82
Fix `PythonDownloadRequest` parsing for partial keys (#12925)
In #12909, I noticed we failed to parse partial download keys with `any`
placeholders. Here, parsing for that is fixed.
2025-04-18 11:25:52 -05:00
Alex Prengère 451c834ebb
Show PyPy downloads during `uv python list` (#12915)
Fixes #12914.
When `PythonDownloadRequest` does not have the `implementation` set, do
not set it to CPython when calling `fill`, otherwise only CPython
interpreters are shown when listing interpreters available for download,
with `uv python list`.
2025-04-17 11:59:13 -05:00
Zanie Blue e58cc39238
Patch `CC` and `CCX` entries in sysconfig for cross-compiled `aarch64` Python distributions (#12239)
Closes https://github.com/astral-sh/uv/issues/12207
2025-04-16 08:34:05 -05:00
John Mumm 278a136bcb
Ensure virtual environment is compatible with interpreter on sync (#12884)
It was possible that a virtual environment became out of sync with the
interpreter it pointed to (for example, if a symlink was changed to an
updated Python version). In such a case, `pyvenv.cfg` and
`activate_this.py` would no longer be correct. This PR detects when the
`version` (`venv` module) or `version_info` (uv and `virtualenv`) field
in `pyvenv.cfg` is out of sync with the interpreter. In such a case, uv
recreates the virtual environment.

Closes #12461
2025-04-15 12:01:14 +02:00
github-actions[bot] 1099867681
Sync latest Python releases (#12784)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-04-09 12:32:00 -05:00
Meitar Reihan 2b62f73064
Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources (#10939)
## Summary

Add an option to overwrite the list of available Python downloads from a
local JSON file by using the environment variable
`UV_PYTHON_DOWNLOADS_JSON_URL`

as an experimental support for providing custom sources for Python
distribution binaries #8015

related #10203

I probably should make the JSON to be fetched from a remote URL instead
of a local file.
please let me know what you think and I will modify the code
accordingly.

## Test Plan

### normal run
```
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.14.0a4+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.14.0a4-linux-x86_64-gnu                 <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu      <download available>
cpython-3.13.1-linux-x86_64-gnu                   <download available>
cpython-3.12.8-linux-x86_64-gnu                   <download available>
cpython-3.11.11-linux-x86_64-gnu                  <download available>
cpython-3.10.16-linux-x86_64-gnu                  <download available>
cpython-3.9.21-linux-x86_64-gnu                   <download available>
cpython-3.8.20-linux-x86_64-gnu                   <download available>
cpython-3.7.9-linux-x86_64-gnu                    <download available>
pypy-3.10.14-linux-x86_64-gnu                     <download available>
pypy-3.9.19-linux-x86_64-gnu                      <download available>
pypy-3.8.16-linux-x86_64-gnu                      <download available>
pypy-3.7.13-linux-x86_64-gnu                      <download available>
```

### empty JSON file
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{}
root@75c66494ba8b:/# /code/target/release/uv python list
root@75c66494ba8b:/# 
```

### JSON file with valid version
```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{
  "cpython-3.11.9-linux-x86_64-gnu": {
    "name": "cpython",
    "arch": {
      "family": "x86_64",
      "variant": null
    },
    "os": "linux",
    "libc": "gnu",
    "major": 3,
    "minor": 11,
    "patch": 9,
    "prerelease": "",
    "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
    "sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9",
    "variant": null
  }
}
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.11.9-linux-x86_64-gnu    <download available>
root@75c66494ba8b:/# 
```

### Remote Path

```sh
root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json 
root@75c66494ba8b:/# /code/target/release/uv python list
error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix)
```

---------

Co-authored-by: Aria Desires <aria.desires@gmail.com>
2025-04-07 13:55:00 -04:00
Charlie Marsh 42dcea0ee2
Bump MSRV to 1.84 (#12670)
## Summary

Closes https://github.com/astral-sh/uv/issues/12649.
2025-04-04 11:49:26 -04:00
Zanie Blue ac145d8150
Skip repeated directories in `PATH` when searching for Python interpreters (#12367)
Closes https://github.com/astral-sh/uv/issues/12302

The change is visible in [this
commit](49be22dad9).
2025-04-03 11:13:07 -05:00
Zanie Blue 12e63eb1ed
Report the queried executable path in `uv python list` (#12628)
In an attempt to avoid reporting shims as their resolved
`sys.executable` path, we report the queried executable path instead.
This seems more correct for this command, broadly? Interestingly, it
changes the reported paths for Homebrew Python

<img width="1430" alt="Screenshot 2025-04-02 at 11 05 18 AM"
src="https://github.com/user-attachments/assets/0e1600e8-fb07-40c7-a6d6-56eaeb4b9293"
/>

Closes #9979
2025-04-02 14:24:58 -05:00
github-actions[bot] b253e2c6e2
Sync latest Python releases (#12251)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-03-17 23:13:19 +00:00
Charlie Marsh 72be5ffb25
Always reinstall local source trees passed to `uv pip install` (#12176)
## Summary

This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)

Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.

Closes #12038.
2025-03-17 21:12:21 +00:00
Tim Felgentreff 40efe6119b
Fix GraalPy abi tag parsing and discovery (#12154)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

There were no GraalPy binary wheels were available when uv support was
added, and thus the abi tag was never tested against actual packages.
Now that GraalPy publishes binary wheels to
https://www.graalvm.org/python/wheels/ we noticed the abi tag was
incorrect and the version info incorrectly determined.

## Test Plan

I tested manually:

```
> target/debug/uv venv --python graalpy testvenv
Using GraalPy 3.11.7 interpreter at: /home/tim/.pyenv/versions/graalpy-24.1.1/bin/graalpy
Creating virtual environment at: testvenv
Activate with: source testvenv/bin/activate
> cat <<EOF> uv.toml
> [[index]]
> url = "https://www.graalvm.org/python/wheels/"
> EOF
> target/debug/uv -v pip install psutil
warning: Found both a `uv.toml` file and a `[tool.uv]` section in an adjacent `pyproject.toml`. The `[tool.uv]` section will be ignored in favor of the `uv.toml` file.
DEBUG uv 0.6.6+3 (be8725553 2025-03-13)
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `graalpy-3.11.7-linux-x86_64-gnu` at `/home/tim/dev/uv/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.11.7 environment at: .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: psutil
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.7
DEBUG Solving with target Python version: >=3.11.7
DEBUG Adding direct dependency: psutil*
DEBUG Found fresh response for: https://www.graalvm.org/python/wheels/psutil/
DEBUG Searching for a compatible version of psutil (*)
DEBUG Selecting: psutil==5.9.8 [compatible] (psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl)
DEBUG No cache entry for: https://gds.oracle.com/download/graalpy-wheels/psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl
DEBUG Tried 1 versions: psutil 1
DEBUG marker environment resolution took 0.968s
Resolved 1 package in 971ms
DEBUG Identified uncached distribution: psutil==5.9.8
DEBUG No cache entry for: https://gds.oracle.com/download/graalpy-wheels/psutil-5.9.8-graalpy311-graalpy241_311_native-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_28_x86_64.whl
Prepared 1 package in 268ms
Installed 1 package in 28ms
 + psutil==5.9.8
DEBUG Released lock at `/home/tim/dev/uv/.venv/.lock`
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-03-13 23:55:07 +00:00
John Mumm 797f1fbac0
Add support for global `uv python pin` (#12115)
These changes add support for

```
uv python pin 3.12 --global 
```

This adds the specified version to a `.python-version` file in the
user-level config directory. uv will now use the user-level version as a
fallback if no version is found in the project directory or its
ancestors.

Closes #4972
2025-03-13 13:48:37 +01:00
Zanie Blue 553bcccb6a
Add support for dynamic musl Python distributions on x86-64 Linux (#12121)
Following the upstream release and #12120, removes gating preventing
installation of the managed musl Python versions.

Of note

- The filtering of musl Python distributions has moved from the Rust
runtime to the metadata fetcher
- The filtering is now conditional on the PBS release date, removing all
old static musl distributions
- We could support the `+static` musl downloads in the future; right
now, they are deprioritized when selecting a variant
- I added test to CI which uses Alpine and installs numpy
2025-03-11 18:14:10 -05:00
github-actions[bot] f3fb1c5a17
Sync latest Python releases (#12120)
Automated update for Python releases.

---------

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-03-11 20:58:47 +00:00
Ed Morley 3c81e4121b
Fix `include_system_side_packages` typo (#11967)
`s/include_system_side_packages/include_system_site_packages/`
2025-03-05 00:49:35 +00:00
github-actions[bot] f5acd60ac9
Fix PyPy Python version label (#11965)
Closes https://github.com/astral-sh/uv/issues/11863

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-03-05 00:36:21 +00:00
Charlie Marsh 3a53ec3c5a
Compare major-minor specifiers when filtering interpreters (#11952)
## Summary

If we're looking at (e.g.) `python3.12`, and we have a `requires-python:
">=3.12.7, <3.13"`, then checking if the range includes `3.12` will
return `false`. Instead, we need to look at the lower- and upper-bound
major-minors of the `requires-python`.

Closes https://github.com/astral-sh/uv/issues/11825.
2025-03-04 23:29:39 +00:00
konsti a169dbeabe
Fix system site packages detection default (#11956)
See https://docs.python.org/3/library/site.html and
a549f43938/Lib/site.py (L614-L639):
The default is for system site packages is true, they have to be
disabled by setting the key to something that is not true.
2025-03-04 18:22:13 -05:00
konsti 12f8d6f2c5
Remove prepended sys.path (#11954)
We prepend the interpreter discovery in a temporary path to `sys.path`,
which we have to strip to avoid the `sys.path` value containing a
then-deleted temp dir.
2025-03-04 16:00:26 +00:00
konsti ceffd7ff80
Discover registry PEP 514 Pythons cross 32/64-bit (#11801)
Fixes #11217

By default, a 64-bit uv does not see a 32-bit global (HKLM) installation
of Python in the registry
(https://github.com/astral-sh/uv/issues/11217). To work around this, we
manually request both 32-bit and 64-bit access using registry access
flags (https://peps.python.org/pep-0514/#sample-code). The flags have no
effect on 32-bit (https://stackoverflow.com/a/12796797/3549270).

This effect is that there is an asymmetry between discovery modes: For
the registry-based discovery using PEP 514, we discover both 32-bit and
64-bit Pythons, while for managed installations, we are stricter and
only discover those matching in bit-ness.

I tested this manually with an additional 32-bit installation of CPython
on a 64-bit machine and windows with 32-bit and 64-bit (x86_64 and i686)
builds of uv.
2025-03-03 14:46:00 +00:00
konsti d712ff243e
Edition 2024 prep: Escape `r#gen` and remove redundant ref (#11922)
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.

In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
2025-03-03 11:13:56 +00:00
github-actions[bot] b5a0b7b2c3
Add pypy3.10-v7.3.19 to managed Python releases (#11814)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-03-02 22:48:35 +00:00
Charlie Marsh 461f4d9007
Make interpreter caching robust to OS upgrades (#11875)
## Summary

In. https://github.com/astral-sh/uv/issues/11857, we had a case of a
user that was seeing incorrect resolution results after upgrading to a
newer version of macOS, since we retained cache information about the
interpreter. This PR adds the OS name and version to the cache key for
the interpreter. This seems to be extremely cheap, and it's nice to make
this robust so that users don't run into the same confusion in the
future.

Closes https://github.com/astral-sh/uv/issues/11857.
2025-03-02 01:36:38 +00:00
Charlie Marsh 3398cb1902
Respect `include-system-site-packages` in layered environments (#11873)
## Summary

We use a similar strategy to the ephemeral overlay: set
`include-system-site-packages` in the `pyvenv.cfg`, and clear it
whenever we access a new environment.

Closes https://github.com/astral-sh/uv/issues/11829.

## Test Plan

Difficult to test because we don't really have support for system
packages in our test infrastructure. But...

```
> uv venv --system-site-packages
> ['', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python313.zip', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/lib-dynload', '/Users/crmarsh/.cache/uv/archive-v0/AhKcORkaCdbBl31VweRtG/lib/python3.13/site-packages', '/Users/crmarsh/workspace/uv/foo/.venv/lib/python3.13/site-packages', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/site-packages']
```

```
> uv venv
> ['', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python313.zip', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13', '/Users/crmarsh/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/lib-dynload', '/Users/crmarsh/.cache/uv/archive-v0/AhKcORkaCdbBl31VweRtG/lib/python3.13/site-packages', '/Users/crmarsh/workspace/uv/foo/.venv/lib/python3.13/site-packages']
```
2025-02-28 22:22:37 -05:00
konsti d136c23d46
Update windows registry to 0.5 (#11799)
This update contains the feature we need to fix #11217
2025-02-26 16:33:31 +01:00
Charlie Marsh c37af945b3
Avoid using owned `String` in deserializers (#11764)
## Summary

This is the pattern I see in a variety of crates, and I believe this is
preferred if you don't _need_ an owned `String`, since you can avoid the
allocation. This could be pretty impactful for us?
2025-02-25 14:28:16 +00:00
samypr100 878497a014
Upgrade Rust toolchain to 1.85 (#11720)
## Summary

* Upgrade the rust toolchain to 1.85.0. This does not increase the MSRV.
* Update windows trampoline to 1.86 nightly beta (previously in 1.85
nightly beta).

## Test Plan

Existing tests
2025-02-23 16:52:34 +01:00
Zanie Blue fdebc0c7fd
Allow use of x86-64 Python on ARM Windows (#11625)
Closes https://github.com/astral-sh/uv/issues/11493
2025-02-19 12:01:46 -06:00
konsti 80b0d4c4e2
Revert: Include environment variables in interpreter info caching (#11622)
Revert #11601 for now

We run Python interpreter discovery with `-I` (#2500) which means these
environments variables are ignored when determining `sys.path`. Unless
we decide to remove the `-I` flag from the `sys.path` query, we
shouldn't release these changes to interpreter discovery caching.
2025-02-19 10:10:21 -06:00
Jo c57fb4dcef
Make fetching github releases faster (#11614)
I noticed that the latest two `sync-python-releases` jobs failed due to
`httpx.RemoteProtocolError: peer closed connection without sending
complete message body (incomplete chunked read)`.

For the current python-build-standalone release, each request page
(defaulting to 30 items per page) takes about 20 seconds and loads
around 32MB of data. This extensive data load might be causing the
request to frequently fail.

In this PR, I reduced number of items per page to 10 and added
`Accept-Encoding: gzip, deflate` to the request header. Now, it takes
about 6 seconds to load, and the compressed response size has been
reduced to 534KB. I hope this would addresses the request failure.
2025-02-19 08:57:56 -06:00
konsti da30cc4ec5
Include environment variables in interpreter info caching (#11601)
We want to use `sys.path` for package discovery (#2500, #9849). For
that, we need to know the correct value of `sys.path`. `sys.path` is a
runtime-changeable value, which gets influenced from a lot of different
sources: Environment variables, CLI arguments, `.pth` files with
scripting, `sys.path.append()` at runtime, a distributor patching
Python, etc. We cannot capture them all accurately, especially since
it's possible to change `sys.path` mid-execution. Instead, we do a best
effort attempt at matching the user's expectation.

The assumption is that package installation generally happens in venv
site-packages, system/user site-packages (including pypy shipping
packages with std), and `PYTHONPATH`. Specifically, we reuse
`PYTHONPATH` as dedicated way for users to tell uv to include specific
directories in package discovery.

A common way to influence `sys.path` that is not using venvs is setting
`PYTHONPATH`. To support this we're capturing `PYTHONPATH` as part of
the cache invalidation, i.e. we refresh the interpreter metadata if it
changed. For completeness, we're also capturing other environment
variables documented as influencing `sys.path` or other fields in the
interpreter info.

This PR does not include reading registry values for `sys.path`
additions on Windows as documented in
https://docs.python.org/3.11/using/windows.html#finding-modules. It
notably also does not include parsing of python CLI arguments, we only
consider their environment variable versions for package installation
and listing. We could try parsing CLI flags in `uv run python`, but we'd
still miss them when Python is launched indirectly through a script, and
it's more consistent to only consider uv's own arguments and environment
variables, similar to uv's behavior in other places.
2025-02-19 10:10:19 +00:00
Jo 0498b9fb1d
Make fetch python downloads deterministic (#11572)
Closes #11549
2025-02-17 08:36:38 -06:00
github-actions[bot] 15d75f91df
Sync latest Python releases (#11527)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-02-15 00:51:47 +00:00
Zanie Blue 81966c43dc
Allow `--python <dir>` requests to match existing environments if `sys.executable` is the same file (#11290)
Closes https://github.com/astral-sh/uv/issues/11288

I tested the reproduction there manually.

I'm a little uncertain about this behavior, it's not true to the spirit
of `--python <dir>` selecting a target environment but this method is
only used to see if an existing environment matches for the purpose of
invalidation in projects and tools where I think we always force a
separate environment anyway?
2025-02-12 12:45:59 -06:00
github-actions[bot] 066fbb74b1
Sync latest Python releases (#11450)
See
https://github.com/astral-sh/python-build-standalone/releases/tag/20250212

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-02-12 10:25:20 -06:00
Charlie Marsh 1cd9c37151
Use stable environments for remote and stdin scripts (#11364)
## Summary

This is a follow-on to #11347 to use a stable directory for remote and
stdin scripts. The annoying piece here was figuring out what to use as
the cache key. For remote scripts, I'm using the URL; for stdin scripts,
there isn't any identifying information, so I'm just using a hash of the
metadata.
2025-02-12 00:54:46 +00:00
konsti ddbc6e3150
Catch broken `mac_ver()` (#11396)
A user reported a homebrew Python that would raise an exception in the
interpreter probing script because `platform.mac_ver()` returned `('',
('', '', ''), '')` on his installation due to
https://github.com/Homebrew/homebrew-core/issues/206778

This is easy enough to catch and show a proper error message instead of
the Python backtrace.
2025-02-10 22:49:16 +01:00
Geoffrey Thomas 25e7209a33
Patch pkg-config files to be relocatable (#11291)
Previously, we patched pkg-config .pc files to have the absolute path to
the directory where we unpack a python-build-standalone release. As
discussed in #11028, we can use ${pcfiledir} in a .pc file to indicate
paths relative to the location of the file itself.

This change was implemented in astral-sh/python-build-standalone#507, so
for newer python-build-standalone releases, we don't need to do any
patching. Optimize this case by only modifying the .pc file if an actual
change is needed (which might be helpful down the line with hard links
or something). For older releases, change uv's patch to match what
python-build-standalone now does.
2025-02-07 17:03:55 -06:00
Geoffrey Thomas 21369326ca
uv-python tests: Use #!/bin/sh instead of #!/bin/bash (#11292)
NixOS has (and POSIX mandates) a /bin/sh but not a /bin/bash, so this
fixes tests on NixOS.
2025-02-07 09:42:33 -06:00
github-actions[bot] 69f1904aeb
Sync latest Python releases (#11318)
Includes https://pypy.org/posts/2025/02/pypy-v7318-release.html

These are labeled as betas in the post but not anywhere obvious to me?
I'm not sure we need to portray this to users.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2025-02-07 09:42:20 -06:00
github-actions[bot] e1fbffb9e0
Update to latest `python-build-standalone` release (#11261)
See https://github.com/astral-sh/python-build-standalone/releases/tag/20250205

---------

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-02-05 17:31:43 -06:00
Charlie Marsh 53d1a7aa6e
Always use base Python discovery logic for cached environments (#11254)
## Summary

This is attempting to solve the same problem surfaced in #11208 and
#11209. However, those PRs only worked for our own managed Pythons. In
Gentoo, for example, they disable the managed Pythons, which led to
failures in the test suite, because the "base Python" returned after
creating a virtual environment would differ from the "base Python" that
you get after _querying_ an existing virtual environment.

The fix here is to apply our same base Python normalization and
discovery logic, to non-standalone / non-managed Pythons. We continue to
use `sys._base_executable` for such Pythons when creating the
virtualenv, but when _caching_, we perform this second discovery step.

Closes https://github.com/astral-sh/uv/issues/11237.
2025-02-05 15:47:56 -05:00
Charlie Marsh 2fad82c735
Set base executable when returning virtual environment (#11209)
## Summary

I'm not sure that this has much of an effect in practice, but currently,
when we return a virtual environment, the `sys_base_executable ` of the
parent ends up being retained as `sys_base_executable` of the created
environment. But these can be, like, subtly different? If you have a
symlink to a Python, then for the symlink, `sys_base_executable` will be
equal to `sys_executable`. But when you create a virtual environment for
that interpreter, we'll set `home` to the resolved symlink, and so
`sys_base_executable` will be the resolved symlink too, in general.
Anyway, this means that we should now have a consistent value between
(1) returning `Virtualenv` from the creation routine and (2) querying
the created interpreter.
2025-02-04 22:32:47 +00:00
Charlie Marsh 34552e2d3d
Use base Python for cached environments (#11208)
## Summary

It turns out that we were returning slightly different interpreter paths
on repeated `uv run --with` commands. This likely didn't affect many (or
any?) users, but it does affect our test suite, since in the test suite,
we use a symlinked interpreter.

The issue is that on first invocation, we create the virtual
environment, and that returns the path to the `python` executable in the
environment. On second invocation, we return the `python3` executable,
since that gets priority during discovery. This on its own is
potentially ok. The issue is that these resolve to different
`sys._base_executable` values in these flows... The latter gets the
correct value (since it's read from the `home` key), but the former gets
the incorrect value (since it's just the `base_executable` of the
executable that created the virtualenv, which is the symlink).

We now use the same logic to determine the "cached interpreter" as in
virtual environment creation, to ensure consistency between those paths.
2025-02-04 17:23:06 -05:00
Zanie Blue ec480bd3ee
Allow discovering virtual environments from the first interpreter found on the `PATH` (#11218)
Closes https://github.com/astral-sh/uv/issues/11214

Special-cases the first Python executable we find on the `PATH`,
allowing it to be considered during searches for virtual environments.

For some context, there are two stages to Python interpreter discovery

1. We find possible Python executables in various sources
2. We query the executables to determine canonical metadata about the
interpreter

We can't really be "sure" if an executable is a complaint virtual
environment during (1), we need to query the interpreter first. This
means that if you're only allowed to installed into virtual
environments, we'll query every interpreter on your PATH. This is not
performant, and causes confusion for users. Notably, I recently improved
error messaging when we can't find any valid interpreters, by showing
the error message we encounter while querying an interpreter (if any).
However, this is problematic when there's an error for an interpreter
that is not relevant to your search. In
https://github.com/astral-sh/uv/pull/11143, I added filtering to avoid
querying additional interpreters, but that regressed some user
experiences where they were relying on us finding implicitly active
virtual environments via the PATH.
2025-02-04 15:41:37 -06:00
Zanie Blue e6ead20aac
Refactor Python discovery iterators to use `filter_ok` and `map_ok` (#11145)
Because now I know those exist
2025-01-31 22:06:23 +00:00
Zanie Blue ba8504fe7a
Filter discovered Python executables by source before querying (#11143)
Closes https://github.com/astral-sh/uv/issues/11138

Though I think we could still have a better error message there.
2025-01-31 21:53:59 +00:00
Charles Tapley Hoyt c6713f5751
Use explicit _GLibCVersion tuple in uv-python crate (#11122) 2025-01-31 11:52:38 +01:00
Zanie Blue 5c0fdfd7ce
Consistently write log messages with capitalized first word (#11111) 2025-01-30 18:56:46 +00:00
Zanie Blue d281f49103
Avoid resolving symbolic links when querying Python interpreters (#11083)
Closes https://github.com/astral-sh/uv/issues/11048

This brings the `PythonEnvironment::from_root` behavior in-line with the
rest of uv Python discovery behavior (and in-line with pip). It's not
clear why we were canonicalizing the path in the first place here.
2025-01-30 10:10:33 -06:00
Zanie Blue 586bab32b9
Update `uv python install --reinstall` to reinstall all previous versions (#11072)
Since we're shipping substantive updates to Python versions frequently,
I want to lower the bar for reinstalling with the latest distributions.

There's a follow-up task that's documented in a test case at
https://github.com/astral-sh/uv/pull/11072/files#diff-f499c776e1d8cc5e55d7620786e32e8732b675abd98e246c0971130f5de9ed50R157-R158
2025-01-30 10:08:06 -06:00
Zanie Blue e26affd27c
Fix best-interpreter lookups when there is an invalid interpreter in the PATH (#11030)
Closes https://github.com/astral-sh/uv/issues/10978

The root cause is the same as #10908 — I should have been more careful
with the original change.
2025-01-28 13:44:32 -06:00
Zanie Blue eeab865194
Update `riscv64` Python downloads to allow install on `riscv64gc` (#10937)
Closes https://github.com/astral-sh/uv/issues/10883
2025-01-24 09:33:29 -06:00
Zanie Blue cbf6d5af9e
Allow fallback to Python download on non-critical discovery errors (#10908)
Closes https://github.com/astral-sh/uv/issues/10898

In #10716, I broke fallback to downloading Python versions by throwing a
different error kind.
2025-01-23 22:37:02 +00:00
konsti db4ab9dc8a
Install and remove managed Python to and from the Windows Registry (PEP 514) (#10634)
## Summary

In preview mode on windows, register und un-register the managed python build standalone installations in the Windows registry following PEP 514.

We write the values defined in the PEP plus the download URL and hash. We add an entry when installing a version, remove an entry when uninstalling and removing all values when uninstalling with `--all`. We update entries only by overwriting existing values, there is no "syncing" involved.

Since they are not official builds, pbs gets a prefix. `py -V:Astral/CPython3.13.1` works, `py -3.13` doesn't.

```
$ py --list-paths                                            
 -V:3.12 *        C:\Users\Konsti\AppData\Local\Programs\Python\Python312\python.exe
 -V:3.11.9        C:\Users\Konsti\.pyenv\pyenv-win\versions\3.11.9\python.exe
 -V:3.11          C:\Users\micro\AppData\Local\Programs\Python\Python311\python.exe
 -V:3.8           C:\Users\micro\AppData\Local\Programs\Python\Python38\python.exe
 -V:Astral/CPython3.13.1 C:\Users\Konsti\AppData\Roaming\uv\data\python\cpython-3.13.1-windows-x86_64-none\python.exe
```

Registry errors are reported but not fatal, except for operations on the company key since it's not bound to any specific python interpreter.

On uninstallation, we prune registry entries that have no matching Python installation (i.e. broken entries).

The code uses the official `windows_registry` crate of the `winreg` crate.

Best reviewed commit-by-commit.

## Test Plan

We're reusing an existing system check to test different (un)installation scenarios.
2025-01-23 14:13:41 +00:00
Sede Soukossi d49be0a7f9
fix(#10717): Omit stdout and stderr sections when empty in errors (#10746)
## Summary

This change closes #10717 

## Test Plan

<!-- How was it tested? -->

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-21 21:45:29 +00:00
Zanie Blue b543881b1b
Show non-critical Python discovery errors if no other interpreter is found (#10716)
Previously, these errors would only be visible in the debug logs as
"Skipping bad interpreter ..." which can lead us to making some
ridiculous claims like "There is no virtual environment" or "Python is
not installed" when really we just failed to query the interpreter for
some reason.

We show the first error, sort of arbitrarily — but I think it matches
user expectation, i.e., this would be the first Python on your PATH.

Related to #10713
2025-01-21 21:11:55 +00:00
Zanie Blue 399086d28f
Disable the distutils setuptools shim during interpreter query (#10819)
Addresses
https://github.com/astral-sh/uv/issues/4204#issuecomment-2604983670

Original context in https://github.com/astral-sh/uv/issues/2302
2025-01-21 18:34:13 +00:00
Charlie Marsh 9799048f92
Omit variant when detecting compatible Python installs (#10722)
## Summary

Closes https://github.com/astral-sh/uv/issues/10586.
2025-01-21 09:01:18 -06:00
Zanie Blue 4f31b44eac
Improve log when distutils is missing (#10713)
See https://github.com/astral-sh/uv/issues/4204 for motivation

This doesn't really reach the user experience I'd expect — i.e., we end
up saying a virtual environment "does not exist" which is a little
silly. However, I think improving the error messaging on interpreter
queries in general should be solved separately. I did one small
"general" change in
89e11d0222
— otherwise we don't show the message at all.

---------

Co-authored-by: konsti <konstin@mailbox.org>
2025-01-20 17:29:29 +00:00
Charlie Marsh c306e46e1d
Remove trailing commas before brackets (#10740) 2025-01-18 19:56:46 +00:00
Charlie Marsh a7f67e877c
Avoid attempting to patch macOS dylib for non-macOS installs (#10721)
## Summary

For example, `cargo run python install
cpython-3.12.8-linux-x86_64_v3-gnu` (on macOS) shouldn't attempt to
patch the dylib. At present, it leads to this warning:

```
warning: Failed to patch the install name of the dynamic library for /Users/crmarsh/.local/share/uv/python/cpython-3.12.8-linux-x86_64_v3-gnu/bin/python3.12. This may cause issues when building Python native extensions.
Underlying error: Failed to update the install name of the Python dynamic library located at `/Users/crmarsh/.local/share/uv/python/cpython-3.12.8-linux-x86_64_v3-gnu/lib/libpython3.12.dylib`
```
2025-01-17 19:21:32 +00:00
Mathieu Kniewallner b46c6db317
docs: fix a few typos (#10675)
## Summary

Fixing a few typos found in the documentation and in comments.
2025-01-16 09:53:59 -05:00
Luca Palmieri 1af02ce8f2
Patch embedded install path for Python dylib on macOS during `python install` (#10629)
## Summary

Fixes #10598 

## Test Plan

Looking for input here @zanieb. How/where would you include tests for
this?
More broadly: do we want a failure to perform the rename to be a hard
error? Or should it start out as a warning?

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-01-15 20:11:54 +00:00
github-actions[bot] ea0ae5e1c6
Sync latest Python releases (#10637)
See
https://github.com/astral-sh/python-build-standalone/releases/tag/20250115

---------

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-01-15 12:55:31 -06:00
Charlie Marsh 5c91217488
Use structured wheel tags everywhere (#10542)
## Summary

This PR extends the thinking in #10525 to platform tags, and then uses
the structured tag enums everywhere, rather than passing around strings.
I think this is a big improvement! It means we're no longer doing ad hoc
tag parsing all over the place.
2025-01-14 01:39:39 +00:00
samypr100 4d3809cc6b
Upgrade Rust toolchain to 1.84.0 (#10533)
## Summary
Upgrade the rust toolchain to 1.84.0. This PR does not bump the MSRV.
2025-01-11 22:19:33 -05:00
konsti 373e34f5dd
Remove `[u64; 4]` from small version to move `Arc` to full version (#10345) 2025-01-07 14:25:32 +00:00
github-actions[bot] eaaf8896ed
Sync latest Python releases (#10333)
Automated update for Python releases.

---------

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-01-06 13:19:35 -06:00
renovate[bot] 5aefe69ecf
Update Rust crate itertools to 0.14.0 (#10316) 2025-01-05 21:27:14 -05:00