## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
It removes the unused result when creating the Cache with the
`from_path` constructor. I don't believe it does any io operations any
more at least.
Add workspace support when using `-r <path>/pyproject.toml` or `-e
<path>` in the pip interface. It is limited to all-editable
static-metadata workspaces, and tests only include a single main
workspace, ignoring path dependencies in another workspace. This can be
considered the MVP for workspace support: You can create a workspace,
you can install from it, but some options and conveniences are still
missing. I'll file follow-up tickets (support in lockfiles, support path
deps in other workspace, #3625)
There is also support in `uv run`, but we need
https://github.com/astral-sh/uv/issues/3700 first to properly support
using different current projects in the bluejay interface, currently the
resolution and therefore the lockfile depends on the current project.
I'd do this change first (it's big enough already), then #3700, and then
add workspace support properly to bluejay.
Fixes#3404
This bumps the versions of pep580 and pep440 to coincide with the
crates.io versions. While not strictly the same, the new types in uv us
an `Inner` struct. Practically I've found I'm still able to use the
patched versions, as can seen from the open PR here:
https://github.com/prefix-dev/pixi/pull/1436.
Would be great if this bump can be done so we can keep combining the
types :)
Follow-up to https://github.com/astral-sh/uv/pull/3797 to clean up the
test isolation in `uv-interpreter`.
I still want to expose a CLI at some point like `uv python <...>` for
discovery and test from there, hopefully this will make that transition
simpler.
Extends #3726
Moves toolchain storage out of `UV_BOOTSTRAP_DIR` (`./bin`) into the
proper user data directory as defined by #3726.
Replaces `UV_BOOTSTRAP_DIR` with `UV_TOOLCHAIN_DIR` for customization.
Installed toolchains will be discovered without opt-in, but the idea is
still that these are not yet user-facing.
## Summary
This also adds filtering for the ARM Pythons, since that needs some libc
changes; and it closes https://github.com/astral-sh/uv/issues/3854 by
way of adding an "arm" branch.
## Summary
Allows, e.g., `UV_SYSTEM_PYTHON=false uv pip install --python
.venv/bin/python`.
This was intended to work after fixing
https://github.com/astral-sh/uv/issues/3000, but I think I misdiagnosed
the scope when closing that issue, and the linked PR there only fixed
some _other_ problems around index URLs.
The only thing we really lose here is we no longer error when
`--break-system-packages` is provided without `--system`, but we can
enforce that elsewhere if we want.
Closes https://github.com/astral-sh/uv/issues/3829.
## Summary
This PR makes a variety of invalid states unrepresentable by changing
`Preference` to require a `PackageName` and `Version`, rather than
accepting a generic `Requirement`. There should be no meaningful
behavior changes.
## Summary
We actually _already_ ignore these (preferences only apply to versions,
not URLs), it just happens later on. This PR thus just avoids crashing.
The behavior is unchanged.
Closes#3822.
Closes#3784
The cache did not use an absolute path. I'm not sure this is actually a
new bug, as this code wasn't touched in #3266 but perhaps there was a
slight difference in the paths we were passing around. Note, just
canonicalizing the path as soon as we see it doesn't work because then
we jump out of the virtual environmnent into the system interpreter.
## Test plan
```
❯ uv venv
Using Python 3.12.3 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ uv pip install anyio
Resolved 3 packages in 81ms
Installed 3 packages in 4ms
+ anyio==4.3.0
+ idna==3.7
+ sniffio==1.3.1
❯ mkdir uv-issue-3784 && cd uv-issue-3784
❯ uv venv
Using Python 3.12.3 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ gcm
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
❯ cargo run -q -- pip list -v -p .venv
DEBUG Checking for Python interpreter in directory `.venv`
TRACE Cached interpreter info for Python 3.12.3, skipping probing: .venv/bin/python3
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
Package Version
------- -------
anyio 4.3.0
idna 3.7
sniffio 1.3.1
❯ cd uv-issue-3784
❯ cargo run -q -- pip list -v -p .venv
DEBUG Checking for Python interpreter in directory `.venv`
TRACE Cached interpreter info for Python 3.12.3, skipping probing: .venv/bin/python3
DEBUG Using Python 3.12.3 environment at /Users/zb/workspace/uv/.venv/bin/python3
Package Version
------- -------
anyio 4.3.0
idna 3.7
sniffio 1.3.1
❯ cd ..
❯ gco zb/fix-relative-venv
Switched to branch 'zb/fix-relative-venv'
❯ cargo run -q -- pip list -v -p .venv
DEBUG Checking for Python interpreter in directory `.venv`
TRACE Cached interpreter info for Python 3.12.3, skipping probing: .venv/bin/python3
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
Package Version
------- -------
anyio 4.3.0
idna 3.7
sniffio 1.3.1
❯ cd uv-issue-3784
❯ cargo run -q -- pip list -v -p .venv
DEBUG Checking for Python interpreter in directory `.venv`
TRACE Cached interpreter info for Python 3.12.3, skipping probing: .venv/bin/python3
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
```
## Summary
Related to https://github.com/astral-sh/uv/issues/3818. We should
_always_ include the package name if we know it's not a file path, even
if it starts with an environment variable.
## Summary
I haven't tested on Windows yet, but the idea here is that we should use
a portable representation when printing paths.
I decided to limit the scope here to paths that we write to output
files.
Closes https://github.com/astral-sh/uv/issues/3800.
## Summary
It turns out that in the
[spec](https://packaging.python.org/en/latest/specifications/binary-distribution-format/#file-name-convention),
if a wheel filename includes a build tag, then we need to use it to
break ties. This PR implements that behavior. (Previously, we dropped
the build tag entirely.)
Closes#3779.
## Test Plan
Run: `cargo run pip install -i https://pypi.anaconda.org/intel/simple
mkl_fft==1.3.8 --python-platform linux --python-version 3.10`. This now
resolves without error. Previously, we selected build tag 63 of
`mkl_fft==1.3.8`, which led to an incompatibility with NumPy. Now, we
select build tag 70.
When parsing requirements from any source, directly parse the url parts
(and reject unsupported urls) instead of parsing url parts at a later
stage. This removes a bunch of error branches and concludes the work
parsing url parts once and passing them around everywhere.
Many usages of the assembled `VerbatimUrl` remain, but these can be
removed incrementally.
Please review commit-by-commit.
## Summary
This seems to be one of the most consistent benchmark cases we have in
terms of standard deviation:
```
➜ hyperfine "target/profiling/main pip compile scripts/requirements/airflow.in" --runs 200
Benchmark 1: target/profiling/main pip compile scripts/requirements/airflow.in
Time (mean ± σ): 292.6 ms ± 6.6 ms [User: 414.1 ms, System: 194.2 ms]
Range (min … max): 282.7 ms … 320.1 ms 200 runs
```
For smaller benchmarks, scispacy and dtlssocket seem to be a bit more
consistent than our current jupyter benchmark, but it hasn't given us
any problems so I'll leave it for now.
e.g. in `uv pip install anyio -v` this message is just noise
```
DEBUG Requirement satisfied: anyio
DEBUG Requirement satisfied: idna>=2.8
DEBUG Requirement satisfied: sniffio>=1.1
DEBUG All editables satisfied:
```
```
DEBUG Acquired lock for `.venv`
```
instead of
```
DEBUG Trying to lock if free: .venv/.lock
```
At trace level, this includes the pre-lock message as well
```
TRACE Checking lock for `.venv`
DEBUG Acquired lock for `.venv`
```
We'll still display the lock file path when something goes wrong
The venv subcommand requires a system interpreter. The tests python path
discovery would previously allow a venv interpreter, failing the venv
tests that don't have system interpreter anymore.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Allows requesting additional transitive dependencies when running a
tool.
e.g. `uv tool run -v --with anyio ruff check example.py` (Why would you
want anyio with ruff? Who knows 😄)
The motivation for doing this now is that I think the first
implementation of `uv tool install` might just shim into `uv tool run`
with pinned dependencies? Regardless this is something we need in the
long run and is a trivial addition right now.
## Summary
We now show yanks as part of the resolution diagnostics, so they now
appear for `sync`, `install`, `compile`, and any other operations.
Further, they'll also appear for cached packages (but not packages that
are _already_ installed).
Closes https://github.com/astral-sh/uv/issues/3768.
Closes#3766.
We usually infer the package the tool is pulled from to be the same name
as the tool itself, but that's not always the case. This allows users to
provide a custom package.
## Summary
This PR removes most of the code in `project/mod.rs` in favor of the
routines exposed in `pip/operations.rs`.
I think we can do a lot more to add more abstraction here and reduce the
verbosity, but for now it deduplicates a _ton_ of logic. The remaining
logic is just instantiating settings etc.
e.g. this error message is not great
```
❯ uv venv --python 3.12.2
× No interpreter found for Python 3.12.2 in provided path, search path, managed toolchains, or parent interpreter
```
e.g.
```
❯ echo "anyio" | cargo run -q -- pip compile - --python 3.9 -v
DEBUG Searching for interpreter that fulfills Python @ 3.9
DEBUG Found a virtual environment at: /Users/zb/workspace/uv/.venv
DEBUG Using Python 3.9.18 interpreter at bin/cpython-3.9.18-macos-aarch64-none/install/bin/python3 for builds
```
e.g. instead of
```
❯ uv venv --python pypy@3.10
× No interpreter found for pypy 3.10 in search path
```
we say
```
❯ uv venv --python pypy@3.10
× No interpreter found for PyPy 3.10 in search path
```
Closes#2222
Closes https://github.com/astral-sh/uv/issues/2058
Replaces https://github.com/astral-sh/uv/pull/2338
See also https://github.com/astral-sh/uv/issues/2649
We use an environment variable (`UV_INTERNAL__PARENT_INTERPRETER`) to
track the invoking interpreter when `python -m uv` is used. The parent
interpreter is preferred over all other sources (though it will be
skipped if it does not meet a `--python` request or if `--system` is
used and it belongs to a virtual environment). We warn if `--system` is
not provided and this interpreter would mutate system packages, but
allow it.
Previously, we enforced `SystemPython` outside of the interpreter
discovery exclusively with source selection. Now, we perform additional
filtering of interpreters depending on if they are a virtual
environment. This should not change any existing behavior, but will make
it much easier to have consistent behavior in ambiguous cases like
https://github.com/astral-sh/uv/pull/3736#discussion_r1610072262 where a
source could provide either a system interpreter or virtual environment
interpreter.
## Summary
This PR takes the functions used in `pip install`, moves them into a
common module, and then replaces all the `pip sync` logic with calls
into those functions. The net effect is that `pip install` and `pip
sync` share far more code and demonstrate much more consistent behavior.
Closes https://github.com/astral-sh/uv/issues/3555.
## Summary
This PR adds editables using a new source type (`editable+...`), and
then extracts the editables from the lockfile in `uv sync`.
Closes https://github.com/astral-sh/uv/issues/3695.
Otherwise `uv venv --python 3.12` can prefer `.venv/bin/python` over the
system Python (which is always used if you don't provide a `--python`
flag). I would find this confusing as a user.
Updates our executable name searches to support implementation names
i.e. `cpython` and `pypy` and adds support for PyPy.
We might want to _not_ support searching for `cpython` because that's
non-standard?
Adds `--offline` support to `uv tool run` and `uv run` because I needed
it on the airplane today.
I think we should move `--offline` to the global settings like
`--native-tls`.
## Summary
Closes https://github.com/astral-sh/uv/issues/3715.
## Test Plan
```
❯ echo "/../test" | cargo run pip compile -
error: Couldn't parse requirement in `-` at position 0
Caused by: path could not be normalized: /../test
/../test
^^^^^^^^
❯ echo "-e /../test" | cargo run pip compile -
error: Invalid URL in `-`: `/../test`
Caused by: path could not be normalized: /../test
Caused by: cannot normalize a relative path beyond the base directory
```
This is mostly a shorter version of `uv run` that infers a requirement
name from the command. The main goal here is to do the smallest amount
of work necessary to get #3560 started.
Closes#3613
e.g.
```shell
$ uv tool run -- ruff check
warning: `uv tool run` is experimental and may change without warning.
Resolved 1 package in 34ms
Installed 1 package in 2ms
+ ruff==0.4.4
error: Failed to parse example.py:1:5: Expected an expression
example.py:1:5: E999 SyntaxError: Expected an expression
Found 1 error.
```
Updates our Python interpreter discovery to conform to the rules
described in #2386, please see that issue for a full description of the
behavior. Briefly, we now will search for interpreters that satisfy a
requested version without stopping at the first Python executable.
Additionally, if retrieving information about an interpreter fails we
will continue to search for a working interpreter. We also add the
plumbing necessary to request Python implementations other than CPython,
though we do not add support for other implementations at this time.
A major internal goal of this work is to prepare for user-facing managed
toolchains i.e. fetching a requested version during `uv run`. These APIs
are not introduced, but there is some managed toolchain handling as
required for our test suite.
Some noteworthy implementation changes:
- The `uv_interpreter::find_python` module has been removed in favor of
a `uv_interpreter::discovery` module.
- There are new types to help structure interpreter requests and track
sources
- Executable discovery is implemented as a big lazy iterator and is a
central authority for source precedence
- `uv_interpreter::Error` variants were split into scoped types in each
module
- There's much more unit test coverage, but not for Windows yet
Remaining work:
- [x] Write new test cases
- [x] Determine correct behavior around executables in the current
directory
- _Future_: Combine `PythonVersion` and `VersionRequest`
- _Future_: Consider splitting `ManagedToolchain` into local and remote
variants
- _Future_: Add Windows unit test coverage
- _Future_: Explore behavior around implementation precedence (i.e.
CPython over PyPy)
Refactors split into:
- #3329
- #3330
- #3331
- #3332Closes#2386
Instead of saying
> we can conclude that you require==0a0.dev0 and
pandas-stubs==2.0.3.230814 are incompatible.
we'll say
> we can conclude that your requirements and pandas-stubs==2.0.3.230814
are incompatible.
Closes#3710
I'm not sure how to get unit test coverage for this, might look into
that. Ideally we'd skip this branch entirely?
I don't really understand why this only happens on windows clippy and
not on linux too, but as usual, boxing the error variant fixes it.
Fixup for #3585
Add minimal support for workspace discovery, only used for determining
paths in the bluejay commands.
We can now discover the workspace structure, namely that the
`pyproject.toml` of a package belongs to a workspace `pyproject.toml`
with members and exclusion. The globbing logic is inspired by cargo. We
don't resolve `workspace = true` metadata declarations yet.
Pubgrub stores incompatibilities as (package name, version range)
tuples, meaning it needs to clone the package name for each
incompatibility, and each non-borrowed operation on incompatibilities.
https://github.com/astral-sh/uv/pull/3673 made me realize that
`PubGrubPackage` has gotten large (expensive to copy), so like `Version`
and other structs, i've added an `Arc` wrapper around it.
It's a pity clippy forbids `.deref()`, it's less opaque than `&**` and
has IDE support (clicking on `.deref()` jumps to the right impl).
## Benchmarks
It looks like this matters most for complex resolutions which, i assume
because they carry larger `PubGrubPackageInner::Package` and
`PubGrubPackageInner::Extra` types.
```bash
hyperfine --warmup 5 "./uv-main pip compile -q ./scripts/requirements/jupyter.in" "./uv-branch pip compile -q ./scripts/requirements/jupyter.in"
hyperfine --warmup 5 "./uv-main pip compile -q ./scripts/requirements/airflow.in" "./uv-branch pip compile -q ./scripts/requirements/airflow.in"
hyperfine --warmup 5 "./uv-main pip compile -q ./scripts/requirements/boto3.in" "./uv-branch pip compile -q ./scripts/requirements/boto3.in"
```
```
Benchmark 1: ./uv-main pip compile -q ./scripts/requirements/jupyter.in
Time (mean ± σ): 18.2 ms ± 1.6 ms [User: 14.4 ms, System: 26.0 ms]
Range (min … max): 15.8 ms … 22.5 ms 181 runs
Benchmark 2: ./uv-branch pip compile -q ./scripts/requirements/jupyter.in
Time (mean ± σ): 17.8 ms ± 1.4 ms [User: 14.4 ms, System: 25.3 ms]
Range (min … max): 15.4 ms … 23.1 ms 159 runs
Summary
./uv-branch pip compile -q ./scripts/requirements/jupyter.in ran
1.02 ± 0.12 times faster than ./uv-main pip compile -q ./scripts/requirements/jupyter.in
```
```
Benchmark 1: ./uv-main pip compile -q ./scripts/requirements/airflow.in
Time (mean ± σ): 153.7 ms ± 3.5 ms [User: 165.2 ms, System: 157.6 ms]
Range (min … max): 150.4 ms … 163.0 ms 19 runs
Benchmark 2: ./uv-branch pip compile -q ./scripts/requirements/airflow.in
Time (mean ± σ): 123.9 ms ± 4.6 ms [User: 152.4 ms, System: 133.8 ms]
Range (min … max): 118.4 ms … 138.1 ms 24 runs
Summary
./uv-branch pip compile -q ./scripts/requirements/airflow.in ran
1.24 ± 0.05 times faster than ./uv-main pip compile -q ./scripts/requirements/airflow.in
```
```
Benchmark 1: ./uv-main pip compile -q ./scripts/requirements/boto3.in
Time (mean ± σ): 327.0 ms ± 3.8 ms [User: 344.5 ms, System: 71.6 ms]
Range (min … max): 322.7 ms … 334.6 ms 10 runs
Benchmark 2: ./uv-branch pip compile -q ./scripts/requirements/boto3.in
Time (mean ± σ): 311.2 ms ± 3.1 ms [User: 339.3 ms, System: 63.1 ms]
Range (min … max): 307.8 ms … 317.0 ms 10 runs
Summary
./uv-branch pip compile -q ./scripts/requirements/boto3.in ran
1.05 ± 0.02 times faster than ./uv-main pip compile -q ./scripts/requirements/boto3.in
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
konsti
Tim de Jager
Charlie Marsh
Tim de Jager
Zanie Blue
Ibraheem Ahmed
Nyakku Shigure
Ofek Lev