## Summary
`pip` uses `sysconfig` for Python 3.10 and later by default; however, it
falls back to `distutils` for earlier Python versions, and distros can
actually tell `pip` to continue falling back to `distutils` via the
`_PIP_USE_SYSCONFIG` variable.
By _always_ using `sysconfig`, we're doing the wrong then when
installing into some system Pythons, e.g., on Debian prior to Python
3.10.
This PR modifies our logic to mirror `pip` exactly, which is what's been
recommended to me as the right thing to do.
Closes https://github.com/astral-sh/uv/issues/2113.
## Test Plan
Most notably, the new Debian tests pass here (which fail on main:
https://github.com/astral-sh/uv/pull/2144).
I also added Pyston as a second stress-test.
## Summary
This PR migrates our virtualenv creation from a setup that assumes prior
knowledge of the correct paths, to a technique borrowed from
`virtualenv` whereby we use `sysconfig` and `distutils` to determine the
paths. The general trick is to grab the expected paths with `sysconfig`,
then make them all relative, then make them absolute for a given
directory.
Closes#2095.
Closes#2153.
## Summary
This will make it easier to use the paths returned by `distutils.py`
(for some cases). No code or behavior changes; just removing some fields
we don't need.
## Summary
When I install via the Windows Store, `interpreter.base_prefix` contains
a bunch of resolved information that leads to a broken environment.
Instead, we now use `sys._base_executable` on Windows by default,
falling back to `sys.base_prefix` if it doesn't exist. (There are some
issues with `sys.base_executable` that lead to complexity in
`virtualenv`, but they only affect POSIX.) Admittedly, I don't know when
`sys._base_executable` wouldn't exist. It exists in all the environments
I've tested.
Additionally, we use the system interpreter directly if we're outside of
a virtualenv.
## Summary
This PR adds a `--python` flag that allows users to provide a specific
Python interpreter into which `uv` should install packages. This would
replace the `VIRTUAL_ENV=` workaround that folks have been using to
install into arbitrary, system environments, while _also_ actually being
correct for installing into non-virtual environments, where the bin and
site-packages paths can differ.
The approach taken here is to use `sysconfig.get_paths()` to get the
correct paths from the interpreter, and then use those for determining
the `bin` and `site-packages` directories, rather than constructing them
based on hard-coded expectations for each platform.
Closes https://github.com/astral-sh/uv/issues/1396.
Closes https://github.com/astral-sh/uv/issues/1779.
Closes https://github.com/astral-sh/uv/issues/1988.
## Test Plan
- Verified that, on my Windows machine, I was able to install `requests`
into a global environment with: `cargo run pip install requests --python
'C:\\Users\\crmarsh\\AppData\\Local\\Programs\\Python\\Python3.12\\python.exe`,
then `python` and `import requests`.
- Verified that, on macOS, I was able to install `requests` into a
global environment installed via Homebrew with: `cargo run pip install
requests --python $(which python3.8)`.
## Summary
Fixes https://github.com/astral-sh/uv/issues/1693
`uv` currently fails when a user has `python` 2 or older installed on
their system without a `python3` or `python3.exe` on their path because
the `get_interpreter_info.py` script fails executing (it uses some
Python 3+ APIs).
This PR fixes this by:
* Returning an explicit error code in `get_interpreter_info` if the
Python version isn't supported
* Skipping over this error in `python_query` if the user requested ANY
python version or a version >= 3.
* Error if the user requested a Python 2 version.
## Test Plan
Error if the user requests a legacy python version.
```
uv venv -p 2
× Python 2 or older is not supported. Please use Python 3 or newer.
```
Ignore any python 2 installation when querying newer python
installations (using v4 here because I have python3 on the path and that
takes precedence over querying python)
```
uv_interpreter::python_query::find_python selector=Major(4)
0.005541s 0ms DEBUG uv_interpreter::interpreter Detecting markers for: /home/micha/.pyenv/shims/python
0.059730s 54ms DEBUG uv_interpreter::python_query Found a Python 2 installation that isn't supported by uv, skipping.
0.059983s 54ms DEBUG uv_interpreter::interpreter Using cached markers for: /usr/bin/python
× No Python 4 In `PATH`. Is Python 4 installed?
```
(This PR message is mostly copied from the comment in the code.)
For local builds of Python, at time of writing, the version numbers end
with
a `+`. This makes the version non-PEP-440 compatible since a `+`
indicates
the start of a local segment which must be non-empty. Thus, `uv` chokes
on it
and [spits out an error][1] when trying to create a venv using a "local"
build
of Python. Arguably, the right fix for this is for [CPython to use a
PEP-440
compatible version number][2].
However, as a work-around for now, [as suggested by pradyunsg][3] as one
possible direction forward, we strip the `+`.
This fix does unfortunately mean that one [cannot specify a Python
version
constraint that specifically selects a local version][4]. But at the
time of
writing, it seems reasonable to block such functionality on this being
fixed
upstream (in some way).
Another alternative would be to treat such invalid versions as strings
(which
is what PEP-508 suggests), but this leads to undesirable behavior in
this
case. For example, let's say you have a Python constraint of `>=3.9.1`
and
a local build of Python with a version `3.11.1+`. Using string
comparisons
would mean the constraint wouldn't be satisfied:
>>> "3.9.1" < "3.11.1+"
False
So in the end, we just strip the trailing `+`, as was done in the days
of old
for [legacy version numbers][5].
I tested this fix by manually confirming that
uv venv --python local/python
failed before it and succeeded after it.
Fixes#1357
[1]: https://github.com/astral-sh/uv/issues/1357
[2]: https://github.com/python/cpython/issues/99968
[3]:
https://github.com/pypa/packaging/issues/678#issuecomment-1436033646
[4]: https://github.com/astral-sh/uv/issues/1357#issuecomment-1947645243
[5]:
085ff41692/packaging/version.py (L168-L193)
First, replace all usages in files in-place. I used my editor for this.
If someone wants to add a one-liner that'd be fun.
Then, update directory and file names:
```
# Run twice for nested directories
find . -type d -print0 | xargs -0 rename s/puffin/uv/g
find . -type d -print0 | xargs -0 rename s/puffin/uv/g
# Update files
find . -type f -print0 | xargs -0 rename s/puffin/uv/g
```
Then add all the files again
```
# Add all the files again
git add crates
git add python/uv
# This one needs a force-add
git add -f crates/uv-trampoline
```
Charlie Marsh
Micha Reiser
Andrew Gallant
Zanie Blue