## Summary
This PR gets rid of the global `--isolated` flag (which serves a bunch
of independent responsibilities right now) on `uv tool run` in favor of
a dedicated `--isolated` flag, which tells uv to avoid re-using an
existing tool environment for this invocation. We'll add the same thing
to `uv run`, to avoid using the base project environment.
This will become a bit clearer in #5466, when we deprecate the
`--isolated` flag on the preview APIs.
## Summary
The idea here is that we hide all resolver output (the grayed out
resolver messages, plus the list of environment modifications) by
default in `uv run` and `uv tool run`. You can pass `--show-resolution`
to re-enable them.
Closes https://github.com/astral-sh/uv/issues/5458.
## Summary
Right now, `--isolated` is read from `uv run` and `uv init` to avoid
discovering the current workspace (or project). This PR moves that
behavior to a dedicated `--no-workspace` flag for `uv init`, and
`--no-project` for `uv run`. They could use the same flag, but
`--no-project` feels confusing for `uv init`, and `--no-workspace` seems
confusing for `uv run` (especially so once you read the documentation,
where we refer to the thing you're omitting as the project).
Closes https://github.com/astral-sh/uv/issues/5429.
This PR represents a different approach to marker propagation in an
attempt to unblock #4640. In particular, instead of propagating markers
when forks are created, we wait until resolution is complete to
propagate all markers to all dependencies in each fork. This ends up
being both more robust (we should never miss anything) and simpler to
implement because it doesn't require mutating a `PubGrubPackage` (which
was pretty annoying). I think the main downside here is that this can
sometimes add markers where they aren't needed.
This actually winds up making quite a few snapshot changes. I went
through each of them. Some of them look like legitimate bug fixes. Some
of them look like superfluous additions. And some of them look like they
would be removed if we had perfect marker normalization. But I don't
think any of the changes are _wrong_.
## Summary
If we just created an entrypoint script, we can of course set the
permissions (we just created it). However, if we're copying from the
cache, we might _not_ own the file. In that case, if we need to change
the permissions (we shouldn't, since the script is likely already
executable -- we set the permissions when we unzip, but I guess they
could _not_ be properly set in the zip itself), we have to copy it.
Closes https://github.com/astral-sh/uv/issues/5581.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
use windows-sys bindings maintained by microsoft devs. winapi didn't has
any updates for more than 3 years
## Test Plan
cargo test. it failed locally because I don't have Python 3.12 installed
## Summary
uv run --directory <path> means that one doesn't have to change to a
project's directory to run programs from it. It makes it possible to use
projects as if they are tool installations.
To support this, first the code reading .python-version was updated so
that
it can read such markers outside the current directory. Note the minor
change this causes (if I'm right), described in the commit.
## Test Plan
One test has been added.
## --directory
Not sure what the name of the argument should be, but it's following uv
sync's directory for now.
Other alternatives could be "--project". Uv run and uv tool run should
probably find common agreement on this (relevant for project-locked
tools).
I've implemented this same change in Rye, some time ago, and then we
went
with --pyproject `<`path to pyproject.toml file`>`. I think using
pyproject.toml file path and not directory was probably a mistake, an
overgeneralization one doesn't need.
Every packse version update is currently causing a huge diff (the size
of the `lock_scenarios.rs` diff in this PR). By redacting the version
from the snapshots, we will only have the actual change in the diff and
not the redundant version change noise.
The second commit moves all remaining packse url arg values to
`common/mod.rs`, which acts as a single source of truth for the packse
version.
## Summary
The package was being installed as editable, but it wasn't marked as
such in `uv pip list`, as the `direct-url.json` was wrong.
Closes https://github.com/astral-sh/uv/issues/5543.
## Summary
The idea here is similar to what we do for wheels: we create the
`CachedEnvironment` in the `archive-v0` bucket, then symlink it to its
content-addressed location. This ensures that we can always recreate
these environments without concern for whether anyone else is accessing
them.
Part of the challenge here is that we want the virtual environments to
be relocatable, because we're now building them in one location but
persisting them in another. This requires that we write relative (rather
than absolute) paths to scripts and entrypoints. The main risk with
relocatable virtual environments is that the scripts and entrypoints
_themselves_ are not relocatable, because they use a relative shebang.
But that's fine for cached environments, which are never intended to
leave the cache.
Closes https://github.com/astral-sh/uv/issues/5503.
## Summary
Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:
* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
`.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.
The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).
Fixes: #3863
## Test Plan
* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
## Summary
This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.
Part of: https://github.com/astral-sh/uv/issues/5263.
## Test Plan
For example:
```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
## Summary
Closes#2187.
The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.
It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.
## Test Plan
Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`
(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.
## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'
Usage: uv.exe pip list --editable
For more information, try '--help'.
```
Consider the following packse scenario:
```toml
[root]
requires = [
"a>=1.0.0 ; python_version < '3.10'",
"a>=1.1.0 ; python_version >= '3.10'",
"a>=1.2.0 ; python_version >= '3.11'",
]
[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```
On current `main`, this produces a dependency on `a` that looks like
this:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```
But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a" },
]
```
That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.
The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.
While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.
For the packse scenario test: https://github.com/astral-sh/packse/pull/206
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.
This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.
In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.
So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.
This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.
This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.
Fixes#5086
Partially addresses #4732
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
## Summary
After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.
This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.
Closes https://github.com/astral-sh/uv/issues/5459.
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
## Summary
It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.
This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.
Closes https://github.com/astral-sh/uv/issues/5424.
## Summary
This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).
Now that we support persistent configuration, users should add:
```toml
[tool.uv]
reinstall = ["foo"]
```
If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).
Closes https://github.com/astral-sh/uv/issues/5420.
Two changes split out from the instability work:
* Break `ResolutionGraph::from_state` into methods before adding new
logic to it.
* `ResolutionGraph`: Convert `NodeKey` type to `PackageRef` struct:
Another small refactoring to make subsequent changes easier.
No functional changes.
@BurntSushi I hope this doesn't interfere with your work too much, the
`PackageRef` should at least make debugging panics here easier.
In preparation for the preferences changes with forking, change the
method structure in `CandidateSelector`. Split out into its own PR to
avoid merge conflicts with main. No functional changes.
Consider these requirements from pylint 3.2.5:
```
Requires-Dist: dill >=0.3.6 ; python_version >= "3.11"
Requires-Dist: dill >=0.3.7 ; python_version >= "3.12"
```
We will split on the python version, but then we may pick a version of
`dill` that's `>=0.3.7` in both branches and also have an otherwise
identical resolution in both forks. In this case, we merge both forks
and store only their conjoined markers.
## Summary
Normalize the order of marker expressions on construction. This removes
the distinction between expressions like `os_name == 'Linux'` vs.
`'Linux' == os_name` throughout the codebase. One caveat here is that
the `in` operator does not have a direct inverse, so we introduce
`MarkerOperator::Contains` to handle that case.
I wanted to land this smaller change before some more intrusive changes
as it simplifies the existing code quite a bit.
## Summary
Prior to this change, the resolver would panic if we ran with
`--offline` and `--no-deps` and we had cached metadata for a _package_
(i.e., the versions) but no cached metadata for the _distribution_
(i.e., the specific wheel), since we weren't validating that the
returned metadata in the `--no-deps` case was actually successful. (We
need metadata, even for `--no-deps`, so that we can validate extras.)
## Test Plan
The added test panics on the previous branch.
## Summary
Users can now run `uv cache prune --ci` (open to feedback on the name of
that flag) to remove all pre-built wheels from the cache, leaving behind
zipped, built wheels (which tend to be the most expensive assets to
re-create). This should greatly increase cache performance in CI
environments, since uploading unzipped wheels can actually hurt
performance if you're persisting the uv cache.
Closes https://github.com/astral-sh/uv/issues/5282.
The test asserts that 28 files were removed. But on my system, 27 files
are removed.
This PR is first about debugging what the difference is (since CI
presumably passes with the status quo snapshot). And then I'm thinking
the right way to fix the test failure is with a filter that replaces the
specific number of files removed (limited to what we know to be
correct) with a placeholder.
## Summary
This is surprisingly complex because we need to decide what happens if
you run `uv run` from within a hidden folder, etc. For now, I did the
simplest thing: we just ignore workspace members that are hidden
directories if they lack a `pyproject.toml`, so you can still include
hidden members, they're just ignored if they don't seem to be projects.
Closes https://github.com/astral-sh/uv/issues/5403.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Implements #5340
## Test Plan
<!-- How was it tested? -->
## Summary
This is a bit simpler than #5333, but seems to work in my testing on
macOS and Windows. It's based on implementations that I found in
[Pixi](36f1bb297d/src/cli/exec.rs (L99))
and
[Wasmer](49e60af8df/lib/wasix/src/state/builder.rs (L1058)).
Closes https://github.com/astral-sh/uv/issues/5257.
## Test Plan
On both macOS and Windows:
- `cargo run -- tool run --from jupyterlab jupyter-lab` -- hit Ctrl-C;
verify that the process exits and the terminal is left in a good state.
- `cargo run -- run python` -- hit Ctrl-C; verify that the process does
_not_ exit, but does on Ctrl-D.
## Summary
Closes https://github.com/astral-sh/uv/issues/5359.
## Test Plan
Unfortunately, the only packages I know of that use this are Ruff and
uv, and both are too heavy to install in a recurring test, so:
`uv tool install hatch==1.12.0 --with uv==0.2.27 --force
--link-mode=symlink`
> DEBUG Found `cpython-3.12.1-macos-aarch64-none` at
`/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
Instead of `<implementation> <version>`
> DEBUG Found cpython 3.12.1 at `/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
## Summary
Prefers, in order:
- The major-minor version of an interpreter discovered via `--python`.
- The `requires-python` from the workspace.
- The major-minor version of the default interpreter.
If the `--python` request is a version or a version range, we use that
without fetching an interpreter.
Closes https://github.com/astral-sh/uv/issues/5299.
The `RequirementsTxtComparator` was written assuming there is one
distribution per package name. This changed with the universal
resolution, which allows multiple versions or urls for the same package
name. The sorting we emitted for these new entries was incidental.
With this change, we properly sort these entries by name, version and
then url in universal mode.
This is an output format change for `--universal` users.
## Summary
This PR avoids an `Invalid package name` error that occurs when using
`uv init .`. This is achieved by slightly reorganizing the code block to
determine the name after the path is canonicalized. The dot path is
expanded to the current directory, and the `file_name` then works as
expected.
Resolve#5329 .
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
1. Fixes errors from https://github.com/astral-sh/uv/pull/4878
2. More cleanup. Removed the need for `MaybeUninit` and `SizeOf`
helpers. Renamed main entrypoint to expected default in windows
`mainCRTStartup` to avoid re-declaring /ENTRY in build.rs.
3. Adds a small basic test harness that >>on windows<< will generate
both types of launchers and run them. I've had been using this locally
to test changes and edge cases, but it might be useful for others. It's
based on core parts of install-wheel-rs.
## Test Plan
Tested locally on a couple of script/gui apps.
---------
Co-authored-by: konsti <konstin@mailbox.org>
## Summary
Similiar to https://github.com/astral-sh/rye/pull/680, I have made a
major refactor to the `fetch-download-metadata.py` script.
Some notable changes:
- Use PEP 723 inline scripts
- Fully type annotated the script
- Implemented async HTTP fetching
- Introduced a `Finder` base class and move finder logic under
`CPythonFinder` subclass, which will make it easier to add a
`PyPyFinder` later.
- Instead of fetching `xxx.sha256` for each file, the script now fetches
a single `SHA256SUMS` file containing checksums for all files in the
release.
As a result, the script now takes around 10 seconds instead of 10+
minutes.
## Plan for Future PRs
- [ ] Implement the `PyPyFinder`
- [ ] Add an GitHub Action to run `fetch-download-metadata.py` daily and
create PR automatically
## Test Plan
```sh
cargo run -- run --isolated -- ./crates/uv-python/fetch-download-metadata.py
```
## Summary
In my setup, I have a directory of wheels symlinked from different
directories. I can point `--find-links` at it with `pip` and it works
but not `uv`.
Currently, `uv` checks if a candidate file `is_file` which is for
regular files. By also checking `is_symlink` I was able to install a
symlinked wheel. I'm not *exactly* sure where, but some other place is
eventually resolving the absolute path of the wheel. (`uv`? The OS?)
## Test Plan
Manually tested - I didn't see any tests for `FlatIndexClient` in the
`uv-client` crate.
```
mkdir /tmp/a /tmp/b # Create a directory of wheels (/tmp/a) and a directory of symlinked wheels (/tmp/b)
cp test-0.0.1-py3-none-any.whl /tmp/a # Add a wheel to the directory of wheels
ln -s /tmp/a/test-0.0.1-py3-none-any.whl /tmp/b/ # Create a symlink to that wheel
uv pip install test --find-links /tmp/b # Install pointing at the symlinked wheels directory
```
## Summary
Otherwise, if the path is already a member, discovery fails.
Also adds a failing test for "adding members that are already covered by
`members`".
## Summary
Excellent find from @konstin. If we have a package that's included in
two forks at the same version, but with different URLs, we need to avoid
collapsing them in the lockfile.
Closes https://github.com/astral-sh/uv/issues/5294.
## Summary
You can still generate instabilities, but at least it's consistent
between including and excluding the extra.
For example, this resolves to 54 and then 52 packages on re-run:
```toml
[project]
name = "transformers"
version = "4.39.0.dev0"
description = "State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow"
requires-python = ">=3.9.0"
dependencies = []
[project.optional-dependencies]
flax = ["jaxlib>=0.4.1,<=0.4.13"]
onnxruntime = ["onnxruntime>=1.4.0"]
ray = ["ray[tune]>=2.7.0"]
deepspeed-testing = [
"dill<0.3.5",
"datasets!=2.5.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
```
I think the difference is just somewhere in PubGrub -- like, we add an
extra dependency, so the iteration order gets changed, and we end up
with a different resolution at the end.
Closes https://github.com/astral-sh/uv/issues/5285.
## Summary
Per https://github.com/astral-sh/uv/pull/5250#issuecomment-2242137762
> It would also be great to have an argument (perhaps leveraging the
global isolated option) that allows us to disable workspace discovery
when we don't want to add a project as a member.
## Test Plan
```sh
$cargo run -- init work
$ cargo run -- init work/pkg-a --isolated
warning: `uv init` is experimental and may change without warning
Initialized project sub-c in /tmp/work
```
Looking at how to merge identical forks, i found that the `Resolution`
can be simplified by treating it as a nodes and edges store (plus pins,
they are separate since they are per name-version, not per
(virtual-)package-version). This should also make #5294 more apparent,
which i didn't touch here.
I additionally added some doc comments to the `Resolution` types.
Our everything-method `solve` tends to grow large, so before i'm adding
more logic, i'm moving some code and some logging statements around to
keep it manageable.
I made minor changes to the logging, otherwise no logic changes, only
refactoring.
## Summary
`dearpygui==1.9.1 has no wheels are available with a matching Python
ABI` is way better than `he requested Python version (>=3.12.3) does not
satisfy Python>=3.12.3`.
Closes https://github.com/astral-sh/uv/issues/5284.
This fixes resolving packages that publish an invalid stub to pypi, such
as tensorrt-llm.
## Summary
In https://github.com/astral-sh/uv/pull/3138 , we implemented
`unsafe-best-match`. However, it seems to not quite work as expected.
When multiple indices contain the same version, it's not clear which
index the current code uses. This PR fixes that to use the first index
the package is in.
## Test Plan
```console
$ echo 'tensorrt-llm==0.11.0' | ./target/debug/uv pip compile - --extra-index-url https://pypi.nvidia.com --python-version=3.10 --index-strategy=unsafe-best-match --annotation-style=line
```
## Summary
Given `Requires-Python: ">=3.12.3"`, we were rejecting wheels like
`dearpygui-1.11.1-cp312-cp312-win_amd64.whl`, since `3.12.0` is not
included in `>=3.12.3`. We instead need to test against the major-minor
version of `Requires-Python`.
The easiest way to do this, I think, is the use the `RequiresPython`
struct, which has a single bound that we can truncate the major-minor.
This also means that we now allow
`dearpygui-1.11.1-cp312-cp312-win_amd64.whl` for specifiers like
`Requires-Python: "==3.10.*"`. This is incorrect on the surface, but it
does match our semantics for `Requires-Python` elsewhere: we treat it as
a lower bound.
Closes https://github.com/astral-sh/uv/issues/5287.
Emulate `pip`'s behaviour and find `pythonw` executable by doing an
`s/python/pythonw/g` style transformation, as opposed to assuming a
constant `pythonw.exe` path.
See #5256 for more detail e.g. why this is a useful behaviour to
emulate.
Fixes: #5256
## Summary
This PR modifies the lockfile to include the impactful resolution
settings, like the resolution and pre-release mode. If any of those
values change, we want to ignore the existing lockfile. Otherwise,
`--resolution lowest-direct` will typically have no effect, which is
really unintuitive.
Closes https://github.com/astral-sh/uv/issues/5226.
## Summary
I don't think that "always reinstall" is tenable for `uv run`. My
perspective on this is that if you want "always reinstall", you can now
set it persistently in your `pyproject.toml` or `uv.toml`.
As a smaller change, we could instead disable this _only_ for the
Project API.
Closes https://github.com/astral-sh/uv/issues/4946.
Resolves#4467.
## Summary
This PR implements the following
1. Add `tool.uv.constraint-dependencies` to pyproject.toml
1. Support to refer `tool.uv.constraint-dependencies` in `uv lock`
1. Support to refer `tool.uv.constraint-dependencies` in `uv pip
compile/install`
These are analogues of the override features implemented in #3839 and
#4369.
## Test Plan
Add test.
## Summary
resolves https://github.com/astral-sh/uv/issues/5217
## Test Plan
existing tests pass (should be perfectly backwards compatible) + added a
few tests to cover the new functionality. in particular, in addition to
the simple use of `--show-version-specifiers`, its interaction with
`--invert` and `--package` flags are tested.
## Summary
This fixes a few bugs introduced by
https://github.com/astral-sh/uv/pull/5104. I previously thought we could
track conflicting locals the same way we track conflicting URLs in
forks, but it turns out that ends up being very tricky. URL forks work
because we prioritize directly URL requirements. We can't prioritize
locals in the same way without conflicting with the URL prioritization
(this may be possible but it's not trivial), so we run into issues where
a correct resolution depends on the order in which dependencies are
traversed.
Instead, we track local versions across all forks in `Locals`. When
applying a local version, we apply all locals with markers that
intersect with the current fork. This way we end up applying some local
versions without creating a fork. For example, given:
```
// pyproject.toml
dependencies = [
"torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
]
// requirements.in
torch==2.0.0
.
```
We choose `2.0.0+cu118` in all cases. However, if a disjoint fork is
created based on local versions, the resolver will choose the most
compatible local when it narrows to a specific fork. Thus we correctly
respect local versions when forking:
```
// pyproject.toml
dependencies = [
"torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
"torch==2.0.0+cpu ; platform_machine != 'x86_64'"
]
// requirements.in
torch==2.0.0
.
```
We should also be able to use a similar strategy for
https://github.com/astral-sh/uv/pull/5150.
## Test Plan
This fixes https://github.com/astral-sh/uv/issues/5220 locally for me,
as well as a few other bugs that were not reported yet.
## Summary
The current code was checking every constraint against every
requirement, regardless of whether they were applicable. In general,
this isn't a big deal, because this method is only used as a fast-path
to skip resolution -- so we just had way more false-negatives than we
should've when constraints were applied. But it's clearly wrong :)
## Test Plan
- `uv venv`
- `uv pip install flask`
- `uv pip install --verbose flask -c constraints.txt` (with `numpy<1.0`)
Prior to this change, Flask was reported as not satisfied.
## Summary
Implements the `uv init` command, which initializes a project
(`pyproject.toml`, `README.md`, `src/__init__.py`) in the current
directory, or in the given path. `uv init` also does workspace
discovery.
Resolves https://github.com/astral-sh/uv/issues/1360.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Spotted some issues in the settings documentation, and room for small
improvements by linking to PEPs/RFCs.
Also updating contribution documentation to mention that it's necessary
to enable the virtual environment before running `mkdocs serve`.
## Test Plan
Running documentation locally.
Fix#5211
## Summary
Change to show only the python installed on the system if
`--python-preference only-system` is specified.
Below is an example of running the command before the change, showing
Python not installed on the system.
#### Before
```bash
# Check system python
$ uv python --preview list --python-preference only-system
cpython-3.12.4-linux-x86_64-gnu <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.11.9-linux-x86_64-gnu <download available>
cpython-3.10.14-linux-x86_64-gnu <download available>
cpython-3.9.19-linux-x86_64-gnu <download available>
cpython-3.8.19-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
```
This PR changes the display to show only Python installed on the system.
#### After
```bash
$ cargo run python --preview list --python-preference only-system
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
Running `target/debug/uv python list --python-preference only-system --preview`
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /bin/python3
```
## Test Plan
- `cargo run python list --python-preference only-system` in Ubuntu
24.04 to verify the display.
- `cargo run python list` in Ubuntu 24.04 to verify that the results
before and after the change were the same.
## Summary
This ensures that we process Python installs and uninstalls as soon as
they complete, rather than waiting for them all to complete, then
processing them sequentially. In practice, it shouldn't be much of a
difference (since the processing is code is fairly light), but it
strikes me as more correct.
## Summary
Addressing this [issue](https://github.com/astral-sh/uv/issues/5147) by
adding the capability for Symbolic linking as a link mode when
installing or syncing dependencies.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Currently, `uv` refuses to install anything on GraalPy. This is
currently blocking GraalPy testing with cibuildwheel, since manylinux
includes both `uv` and `graalpy` (but doesn't test with `uv`), whereas
cibuildwheel defaults to `uv`. See e.g.
https://github.com/pypa/cibuildwheel/actions/runs/9956369360/job/27506182952?pr=1538
where it gives
```
+ python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
* Creating isolated environment: venv+uv...
* Using external uv from /usr/local/bin/uv
* Installing packages in isolated environment:
- setuptools >= 40.8.0
> /usr/local/bin/uv pip install "setuptools >= 40.8.0"
< error: Unknown implementation: `graalpy`
```
## Test Plan
I simply based the GraalPy support on PyPy and added some small tests.
I'm open to discussing how to test this. GraalPy is available for
manylinux images and with setup-python, so we should be able to add
tests against it to the CI. I locally confirmed by installing `uv` into
a GraalPy venv and then trying things like `uv pip install Pillow` and
testing those extensions.
## Summary
Closes https://github.com/astral-sh/uv/issues/5124.
## Test Plan
Ran `cargo run -- help pip compile` on my Windows machine, which failed
before but succeeds after this change.
## Summary
You can now use `uv run --locked` to assert that the lockfile doesn't
change, or `uv run --frozen` to run without attempting to update the
lockfile at all.
Closes https://github.com/astral-sh/uv/issues/5185.
As user, you specify a list of extras. Internally, we decompose this
into one virtual package per extra. We currently leak this abstraction
by writing one entry per extra to the lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas" },
{ name = "pandas", extra = "excel" },
{ name = "pandas", extra = "hdf5" },
{ name = "pandas", extra = "html", marker = "os_name != 'posix'" },
{ name = "pandas", extra = "output-formatting", marker = "os_name == 'posix'" },
{ name = "pandas", extra = "plot", marker = "os_name == 'posix'" },
]
```
Instead, we should merge the extras into a list of extras, creating a
more concise lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas", extra = ["excel", "hdf5"] },
{ name = "pandas", extra = ["html"], marker = "os_name != 'posix'" },
{ name = "pandas", extra = ["output-formatting", "plot"], marker = "os_name == 'posix'" },
]
```
The base package is now implicitly included, as it is in PEP 508.
Fixes#4888
## Summary
Makes the `tools()` return value include per-tool errors. This makes it
easy to skip (rather than failing) in `uv tool list`, _and_ improves `uv
tool uninstall` to remove those invalid tools, rather than leaving them
around. (We already had that behavior for `uv tool uninstall ruff` with
an invalid `ruff`, but `uv tool uninstall --all` just left them.)
Closes https://github.com/astral-sh/uv/issues/5151.
Search for all `python3.x` minor versions in PATH, skipping those we
already know we can use.
For example, let's say `python` and `python3` are Python 3.10. When a
user requests `>= 3.11`, we still need to find a `python3.12` in PATH.
We do so with a regex matcher.
Fixes#4709
Warn when there is a direct dependency without a lower bound and
`--resolution lowest` is set.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
"Bare" made sense when we had a variant that seeded the environment, but
now that the crate _only_ creates a bare environment, lets drop that
terminology.
## Summary
Hashes will be validated if present, but aren't required (since, e.g.,
some registries will omit them, as will Git dependencies and such).
Closes https://github.com/astral-sh/uv/issues/5168.
## Summary
This is an alternative to `--require-hashes` which will validate a hash
if it's present, but ignore requirements that omit hashes or are absent
from the lockfile entirely.
So, e.g., transitive dependencies that are missing will _not_ error; nor
will dependencies that are included but lack a hash.
Closes https://github.com/astral-sh/uv/issues/3305.
## Summary
We only need to store one hash -- it should be the "strongest" hash. In
practice, most registries (like PyPI) only serve one, and we only
compute a SHA256 hash for direct URLs.
Part of: https://github.com/astral-sh/uv/issues/4924
## Test Plan
I verified that changing:
```diff
diff --git a/crates/distribution-types/src/hash.rs b/crates/distribution-types/src/hash.rs
index 553a74f55..d36c62286 100644
--- a/crates/distribution-types/src/hash.rs
+++ b/crates/distribution-types/src/hash.rs
@@ -31,7 +31,7 @@ impl<'a> HashPolicy<'a> {
pub fn algorithms(&self) -> Vec<HashAlgorithm> {
match self {
Self::None => vec![],
- Self::Generate => vec![HashAlgorithm::Sha256],
+ Self::Generate => vec![HashAlgorithm::Sha256, HashAlgorithm::Sha512],
Self::Validate(hashes) => {
let mut algorithms = hashes.iter().map(HashDigest::algorithm).collect::<Vec<_>>();
algorithms.sort();
```
Then running `uv lock` with a URL gave me:
```toml
[[distribution]]
name = "iniconfig"
version = "2.0.0"
source = { url = "62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl" }
wheels = [
{ url = "62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha512:44cc53a6c8dd7cf4d6d52bded308bcc4b4f85fff2ed081f60f7d4beaa86a7cde6d099e3976331232d4cbd472ad5d1781064725b0999c7cd3a2a4d42df687ee81" },
]
```
## Summary
It turns out that if `path` is a symlink,
`File::create(path)?.write_all(content.as_ref())?` will overwrite the
_target_ file. That means an entrypoint named `python` would actually
overwrite the user's source Python executable, which is symlinked into
the virtual environment.
This PR replaces that code with our atomic write method.
Closes https://github.com/astral-sh/uv/issues/5152.
## Test Plan
I ran through the test plan
`https://github.com/astral-sh/uv/issues/5152`, but used an executable
named `bar` linked to `foo.txt` instead...
* Use a dedicated `ResolverMarkers` check in the fork state. This is
better than the `MarkerTree::And(Vec::new())` check.
* Report the timing correct naming universal resolution instead of two
spaces around an empty string when there are no markers.
* On resolution error, show the split that we're in. I'm not sure how to
word this, since we're doing a universal resolution until we fork, so
the trace may contain information from requirements that are not part of
this fork.
## Summary
Resolves#5139
`PythonInstallationKey` was sorted as a string, which caused `3.8` to
appear before `3.11`. This update changes the sorting of
`PythonInstallationKey` to be a descending order by version.
## Test Plan
```sh
$ cargo run -- python install 3.8 3.12
$ cargo run -- tool run -v python -V
DEBUG uv 0.2.25
warning: `uv tool run` is experimental and may change without warning.
DEBUG Searching for Python interpreter in managed installations, system path, or `py` launcher
DEBUG Searching for managed installations at `C:\Users\xx\AppData\Roaming\uv\data\python`
DEBUG Found managed Python `cpython-3.12.3-windows-x86_64-none`
DEBUG Found cpython 3.12.3 at `C:\Users\xx\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none\install\python.exe` (managed installations)
DEBUG Using request timeout of 30s
DEBUG Using request timeout of 30s
DEBUG Acquired lock for `C:\Users\nigel\AppData\Roaming\uv\data\tools`
DEBUG Using existing environment for tool `httpx`: C:\Users\xx\AppData\Roaming\uv\data\tools\httpx
DEBUG Using existing tool `httpx`
DEBUG Running `httpx -v`
```
## Summary
Fixes#5082.
Adds a new `Printer::NoProgress` that is identical to `Printer::Default`
but doesn't draw any progress bar.
## Test Plan
It seems to me that as of now it's not possible to use `insta-cmd` to
get any progress bar in the comparable output of command.
Best way to test this would be to run any command that usually shows
progress indicators like `uv pip install` with and without
`--no-progress` options.
## Summary
First part of https://github.com/astral-sh/uv/issues/5093.
Remaining:
- Global settings
- `pip`-specific settings (some will be copied-over from here)
- Auto-generating the "Possible values" for enums
Add support for path dependencies from a package in one workspace to a
package in another workspace, which it self has workspace dependencies.
Say we have a main workspace with packages `a` and `b`, and a second
workspace with `c` and `d`. We have `a -> b`, `b -> c`, `c -> d`. This
would previously lead to a mangled path for `d`, which is now fixed.
Like distribution paths, we split workspace paths into an absolute
install path and a relative (or absolute, if the user provided an
absolute path) lock path.
Part of https://github.com/astral-sh/uv/issues/3943
Fixes a concurrency issue when multiple processes are installing the
same package in different virtual environments from Git ref (not a
specific Git commit).
## Symptoms
That's how some of symptoms looked like in our case:
```
DEBUG uv 0.2.21
DEBUG Checking for Python interpreter at path `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python`
DEBUG Using Python 3.12.4 environment at /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python
DEBUG Acquired lock for `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7`
DEBUG At least one requirement is not satisfied: torch
DEBUG Using request timeout of 300s
DEBUG Found 37 packages in `--find-links` entry: /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/pip/wheels
DEBUG Updating git source `Url { scheme: "https", cannot_be_a_base: false, username: "***", password: None, host: Some(Domain("github.com")), port: None, path: "/iterative/datachain", query: None, fragment: None }`
DEBUG Attempting GitHub fast path for: https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test
DEBUG failed to check github HTTP status client error (404 Not Found) for url (https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test)
DEBUG Performing a Git fetch for: https://***@github.com/iterative/datachain
error: Failed to download and build: `datachain @ git+https://***@github.com/iterative/datachain@fix-distributed-test`
Caused by: Git operation failed
Caused by: process didn't exit successfully: `git clone --local /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/db/9d45a3e6f56b0a69 /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8` (exit status: 128)
--- stderr
fatal: destination path '/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8' already exists and is not an empty directory.
```
## Cause of the issue
It is the same command that is failing - `git clone`, and I think it's
happening because it was trying to first get the repo to dereference the
`fix-distributed-test` branch:
`Given a remote source distribution, return a precise variant, if
possible.`
And it's happening w/i acquiring a lock around cache.
## Fix
I thinks we can reuse the existing `fetch` method that has already lock
around cache:
https://github.com/astral-sh/uv/pull/5051/files#diff-f58bb99dee2c4922d156ace3e7de651f0d9a81fc8e9447a2ad865de5c53543fcR61-R68
```python
// Avoid races between different processes, too.
let lock_dir = cache.join("locks");
....
```
## Questions
- Are there any tests that cover concurrency? I'm quite new to Rust and
if someone can point me to some examples and I can create a similar test
or a new one.
- Is error handling done correctly in this PR (again, I'm new to Rust -
I'll review and read about it, but it's better also for someone else to
review this)
## Summary
Currently, the `Locals` type relies on there being a single local
version for a given package. With marker expressions this may not be
true, a similar problem to https://github.com/astral-sh/uv/pull/4435.
This changes the `Locals` type to `ForkLocals`, which tracks locals for
a given fork. Local versions are now tracked on `PubGrubRequirement`
before forking.
Resolves https://github.com/astral-sh/uv/issues/4580.
Specifically, this shows the resolution produced by the
resolver *before* constructing a resolution graph.
Unlike most trace messages, this is a multi-line message
that needs to do some small amount of work to build
itself. So we do an explicit gating on the log level here
instead of just relying on the `trace!` macro itself.
I messed up the order of checks in #5033, due to which it failed to
exclude the case of `-P package`, as `arg.startswith("-P")` check came
first and skipped only the first argument.
That means that, in the following command:
```console
uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in -P attrs==18.1.0
```
The generated header would exclude `-P`, but keep `attrs==18.1.0`.
```plaintext
# This file was autogenerated by uv via the following command:
# uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in attrs==18.1.0
```
But we want to check for an exact match first and then only check for
the case when option and value are together.
This also affected `--find-links` short option of style `-f <uri>`.
Hopefully, third times going to be a charm. 😳
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
I tested locally, and also changed one snapshot test to use `-P` for
variation. I don't think it's worth an extra test, but can do that for
sure.
## Summary
When range requests aren't supported, we fall back to streaming the
wheel, stopping as soon as we hit a `METADATA` file. This is a small
optimization, but the downside is that we don't get to cache the
resulting wheel...
We don't know whether `METADATA` will be at the beginning or end of the
wheel, but it _seems_ like a better tradeoff to download and cache the
entire wheel?
Closes: https://github.com/astral-sh/uv/issues/5088.
Sort of a revert of: https://github.com/astral-sh/uv/pull/1792.
## Summary
Workaround the `stream_wheel` not retry issue
[found](https://github.com/astral-sh/uv/issues/3514#issuecomment-2229820667)
in #3514, it's not a perfect solution but I think it's acceptable
because the error should not occur frequently.
## Test Plan
Manually using `iptables -A OUTPUT -p tcp -dport 3128 -j REJECT
--reject-with tcp-reset` to inject connection reset error to the HTTP
proxy that proxies PyPI requests.
```
error: Failed to prepare distributions
Caused by: Failed to fetch wheel: piqp==0.4.1
Caused by: Request failed after 3 retries
Caused by: error sending request for url (09ade94dfdd3c368ac505b6ca09831/piqp-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl)
Caused by: client error (Connect)
Caused by: tcp connect error: Connection refused (os error 111)
Caused by: Connection refused (os error 111)
```
## Summary
`echo "torch==1.10.0" | cargo run pip compile - -p 3.12 --no-deps` now
correctly fails. Previously, we were accepting the wheel
`torch-1.10.0-cp36-none-macosx_10_9_x86_64.whl` as compatible with
Python 3.10 due to the `none` ABI.
Closes https://github.com/astral-sh/uv/issues/5085.
## Summary
The example in the linked issue doesn't quite work, but I think it has
to do with the existing filtering logic. Will follow-up separately.
Closes https://github.com/astral-sh/uv/issues/5012.
## Summary
So this PR introduces change to how `Array` of dependencies
representation is reformatted while `PyProjectTomlMut` is manipulated.
These changes are here for it to respect the original indentation.
Closes https://github.com/astral-sh/uv/issues/5009
## Test Plan
Using `pyproject.toml` like
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests"
]
```
Executed
```
$ uv add httpx
```
And expected in `pyproject.toml`
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests",
"httpx",
]
```
Preserving original indentation
This name should lead to less confusion. Unfortunately this is a
"breaking cache change" so everyone's cache will be invalidated. I'm not
sure if we should support a rename-on-upgrade.
edit: We can make the breaking change next time we bump the version
## Summary
Resolves#5017.
Note: This re-uses the same function defined in #5019 to find matching
packages.
## Test Plan
`cargo test`
```console
❯ ./target/debug/uvx --from fastapi fastapi
warning: `uvx` is experimental and may change without warning.
Resolved 33 packages in 427ms
warning: The fastapi executable is not part of the fastapi package. It is provided by the fastapi-cli package. Use `uvx --from fastapi-cli fastapi` instead.
Usage: fastapi [OPTIONS] COMMAND [ARGS]...
Try 'fastapi --help' for help.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Should fix#2092.
This PR changes `uv venv` so it also creates symlinks to `pypy` on Unix
and copies executables on Windows when creating a new environment using
PyPy.
I found a bit of discrepancy between creation of a venv using `python`
and `uv`, as using `python` brings all the executables with it. While
`uv` brings only those without any version number, at least on Windows.
The behaviour is different on Unix as we take the versioned symlinks
too.
Some examples below.
`python -m venv` generates the following `Scripts` folder.
```
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/14/2024 15:41 2031 activate
-a---- 7/14/2024 15:41 1029 activate.bat
-a---- 7/14/2024 15:41 9033 Activate.ps1
-a---- 7/14/2024 15:41 393 deactivate.bat
-a---- 7/14/2024 15:40 27648 libffi-8.dll
-a---- 7/14/2024 15:41 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 15:41 108424 pip.exe
-a---- 7/14/2024 15:41 108424 pip3.10.exe
-a---- 7/14/2024 15:41 108424 pip3.exe
-a---- 7/14/2024 15:41 79360 pypy.exe
-a---- 7/14/2024 15:41 79360 pypy3.10.exe
-a---- 7/14/2024 15:41 79360 pypy3.10w.exe
-a---- 7/14/2024 15:41 79360 pypy3.exe
-a---- 7/14/2024 15:41 79360 pypyw.exe
-a---- 7/14/2024 15:41 79360 python.exe
-a---- 7/14/2024 15:41 79360 python3.10.exe
-a---- 7/14/2024 15:41 79360 python3.exe
-a---- 7/14/2024 15:41 79360 pythonw.exe
```
`uv venv` instead generates this.
```
-a---- 7/14/2024 16:27 3360 activate
-a---- 7/14/2024 16:27 2251 activate.bat
-a---- 7/14/2024 16:27 2627 activate.csh
-a---- 7/14/2024 16:27 4191 activate.fish
-a---- 7/14/2024 16:27 3875 activate.nu
-a---- 7/14/2024 16:27 2766 activate.ps1
-a---- 7/14/2024 16:27 2378 activate_this.py
-a---- 7/14/2024 16:27 1728 deactivate.bat
-a---- 7/13/2024 19:19 27648 libffi-8.dll
-a---- 7/13/2024 19:19 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 16:27 1215 pydoc.bat
-a---- 7/13/2024 19:19 79360 pypy.exe
-a---- 7/13/2024 19:19 79360 pypyw.exe
-a---- 7/13/2024 19:19 79360 python.exe
-a---- 7/13/2024 19:19 79360 pythonw.exe
```
## Test Plan
To verify the correct behaviour:
1. Download and install PyPy from [official
website](https://www.pypy.org/download.html)
2. Call `uv venv -p <path_to_pypy_>`
3. Run `.\.venv\Scripts\activate` on Windows or
`./.venv/Scripts/activate` on Unix
4. Run `pypy`
I thought of writing some automated tests but I couldn't rely on `uv
python install` command to install PyPy as it's not in the list of
installable Python builds.
In some cases, it's possible for the marker expressions on conflicting
dependency specification to be disjoint but *incomplete*. That is, if
one unions the disjoint markers, the result is not the complete set of
marker environments possible. There may be some "gap" of marker
environments not covered by the markers.
This is a problem in practice because, before this commit, we only
created forks in the resolver for specific marker expressions. So if a
dependency happened to fall in a "gap," our resolver would never see it.
This commit fixes this by adding a new split covering the negation of
the union of all marker expressions in a set of forks for a specific
package.
Originally, I had planned on only creating this split when it was known
that the gap actually existed. That is, when the negation of the marker
expressions did *not* correspond to the empty set. After a lot of
thought, unfortunately, this (I believe) effectively boils down to 3SAT,
which is NP-complete.
Instead, what we do here is *always* create an extra split unless we can
definitively tell that it is empty. We look for a few cases, but
otherwise throw our hands up and potentially do wasted work.
This also updates the lock scenario tests to reflect the actual bug fix
here.
An important update here is the new `fork-incomplete-markers` test.
The snapshot saved here is actually wrong; it is the bug we want to
fix. In particular, it is missing a dependency on `c` in its lock file
because it falls into a gap created by incomplete-but-disjoint marker
expressions.
The only pubgrub error that can occur is a `NoSolutionError`, and the
only place it can occur is `unit_propagation`, all other variants if
`PubGrubError` are unreachable. By changing the return type on pubgrub's
side (https://github.com/astral-sh/pubgrub/pull/28), we can remove the
pattern matching and the `unreachable!()` asserts on `PubGrubError`.
Our pubgrub error wrapper used to have a two phased initialization,
first mostly stubs in `solve[_tracked]()` and then adding the actual
context in `resolve()`. When constructing the error in `solve` we
already have all this context, so we can unify this to a regular
constructor and remove the special casing in `resolve()` and `hints()`.
Currently, with
```toml
[project]
name = "transformers"
version = "4.39.0.dev0"
requires-python = ">=3.10"
dependencies = [
"torch==1.10.0"
]
```
i get
```
$ uv sync --preview
Resolved 3 packages in 7ms
error: found distribution torch==1.10.0 @ registry+https://pypi.org/simple with neither wheels nor source distribution
```
This error message is wrong, there are wheels, they are just not
compatible. I initially got this error message during `uv lock` (in a
build), so i also added that this is about installation, not about
locking.
We should reject this version immediately because with the current
requires python, it can never be installed, but even then we need to
change the error message because you can be on the correct python
version, but an unsupported platform.
Fix#4988
## Summary
Running `uv python list` on glibc-based Linux will list musl pythons.
```bash
$ uv version
uv 0.2.24
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.11.9-linux-x86_64-musl <download available>
cpython-3.10.14-linux-x86_64-musl <download available>
cpython-3.9.19-linux-x86_64-musl <download available>
cpython-3.8.19-linux-x86_64-musl <download available>
cpython-3.7.9-linux-x86_64-musl <download available>
```
Change it to show Python matching the environment's libc as follows.
```bash
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.11.9-linux-x86_64-gnu <download available>
cpython-3.10.14-linux-x86_64-gnu <download available>
cpython-3.9.19-linux-x86_64-gnu <download available>
cpython-3.8.19-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
```
Also, if --all-platforms is specified, change to list Python for all
architectures and libc.
```bash
$ uv python list --all-platforms
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-windows-x86_64-none <download available>
cpython-3.12.3-windows-x86-none <download available>
cpython-3.12.3-macos-x86_64-none <download available>
cpython-3.12.3-macos-aarch64-none <download available>
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.12.3-linux-s390x-gnu <download available>
cpython-3.12.3-linux-powerpc64le-gnu <download available>
cpython-3.12.3-linux-armv7-gnueabihf <download available>
cpython-3.12.3-linux-armv7-gnueabi <download available>
cpython-3.12.3-linux-aarch64-gnu <download available>
...
```
## Test Plan
The following commands were executed on the command line to confirm the
results in Ubuntu 24.04.
- `cargo run python list`
- `cargo run python list --all-platforms`
## Summary
Closes https://github.com/astral-sh/uv/issues/5040.
## Test Plan
```
❯ cargo run pip install torch torch-scatter --no-cache
⠼ torch-scatter==2.1.2 error: Failed to download and build `torch-scatter==2.1.2`
Caused by: Failed to build: `torch-scatter==2.1.2`
Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:
--- stderr:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
self.run_setup()
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'torch'
---
Caused by: This error likely indicates that torch-scatter==2.1.2 depends on torch, but doesn't declare it as a build dependency. If torch-scatter==2.1.2 is a first-party package, consider adding torch to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation
```
This excludes `--upgrade-package` from `compile_command` when value and
option are passed as a single argument. Eg:
```console
--upgrade-package=package
-P=package
-Ppackage
```
I missed this on #5032.
Fixes#5031.
## Test Plan
Tested locally
## Summary
Largely based on rustup's implementation (linked in the source).
Closes#5027.
## Test Plan
- Changed the executable directory to `uv/foo`.
- Ran script; verified that I could access executables in `foo`.
## Summary
Use the lockfile to prefill the `InMemoryIndex` used by the resolver.
This enables us to resolve completely from the lockfile without making
any network requests/builds if the requirements are unchanged. It also
means that if new requirements are added we can still avoid most I/O
during resolution, partially addressing
https://github.com/astral-sh/uv/issues/3925.
The main limitation of this PR is that resolution from the lockfile can
fail if new versions are requested that are not present in the lockfile,
in which case we have to perform a fresh resolution. Fixing this would
likely require lazy version/metadata requests by `VersionMap` (this is
different from the lazy parsing we do, the list of versions in a
`VersionMap` is currently immutable).
Resolves https://github.com/astral-sh/uv/issues/3892.
## Test Plan
Added a `deterministic!` macro that ensures that a resolve from the
lockfile and a clean resolve result in the same lockfile output for all
our current tests.
## Summary
Brings in some learnings from `rustup`:
fede22fea7/src/cli/self_update/shell.rs (L197).
For example: we only need to write to `.zshenv` (but we have to respect
`ZDOTDIR`). Additionally, for Fish, we need to respect `XDG_CONFIG_HOME`
## Summary
I'll open follow-up tickets for Windows support.
Closes https://github.com/astral-sh/uv/issues/4953.
## Test Plan
```
❯ cargo run tool install flask
Resolved 7 packages in 353ms
Prepared 7 packages in 392ms
Installed 7 packages in 17ms
+ blinker==1.8.2
+ click==8.1.7
+ flask==3.0.3
+ itsdangerous==2.2.0
+ jinja2==3.1.4
+ markupsafe==2.1.5
+ werkzeug==3.0.3
Installed 1 executable: flask
warning: /Users/crmarsh/.local/bin is not on your PATH. To use installed tools, run:
export PATH="/Users/crmarsh/.local/bin:$PATH"
```
Then:
```
❯ which flask
flask not found
```
Then:
```
❯ cargo run tool ensurepath
warning: `uv tool ensurepath` is experimental and may change without warning.
Updated configuration file: /Users/crmarsh/workspace/puffin/bar
Restart your shell for the changes to take effect.
```
Then:
```
❯ which flask
/Users/crmarsh/.local/bin/flask
```
## Summary
Resolves#5013.
## Test Plan
```console
❯ ./target/debug/uv tool run fastapi-cli
warning: `uv tool run` is experimental and may change without warning.
Resolved 9 packages in 28ms
The executable fastapi-cli was not found.
However, the following executables are available via uv tool run --from fastapi-cli <EXECUTABLE>:
- fastapi
```
```console
❯ ./target/debug/uvx fastapi-cli
warning: `uvx` is experimental and may change without warning.
Resolved 9 packages in 23ms
The executable fastapi-cli was not found.
However, the following executables are available via uvx --from fastapi-cli <EXECUTABLE>:
- fastapi
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Fixes#4941.
This PR adds a `--no-pager` option in `help` command to explicitly
disable the pager.
I noted that the template used for the text printed when calling `help`
with no argument or option doesn't show any option. It made sense before
this PR since `help` didn't have any available option. Though I'm unsure
if it makes sense to update the template as it would make it extremely
verbose as all the global options would be shown too.
I leave the decision to you.
## Test Plan
I ran `cargo run -- help` to verify `--isolated` was visible and it.
I ran clippy with `cargo clippy --workspace --all-targets --all-features
--locked -- -D warnings` as CI does.
I also ran tests locally with:
```
cargo nextest run \
--features python-patch \
--workspace \
--status-level skip --failure-output immediate-final --no-fail-fast -j 12 --final-status-level slow
```
[Doc tests can't use crate internal APIs unfortunately.][internal-doc]
I really want them to be able to, but for now, mark such tests as
`ignore`.
I was motivated to do this because it otherwise breaks `cargo t --all`
for me.
[internal-doc]: https://github.com/rust-lang/rust/issues/50784
## Summary
This PR makes the `--isolated` global argument visible, previously it
was hidden.
Fixes#4981.
## Test Plan
I ran `cargo run -- help` to verify `--isolated` was visible and it is.
I ran clippy with `cargo clippy --workspace --all-targets --all-features
--locked -- -D warnings` as CI does.
I also ran tests locally with:
```
cargo nextest run \
--features python-patch \
--workspace \
--status-level skip --failure-output immediate-final --no-fail-fast -j 12 --final-status-level slow
```
It's unclear to me whether this was intentional or not, but
I realized that converting a MarkerExpression to a string
treated EqualStar and NotEqualStar as Equal and NotEqual,
respectively. I tweaked this to match the Display impl for
VersionSpecifier.
(Negation tests in the next commit cover this change.)
## Summary
Converting to a lock requires that we generate hashes; but generating
hashes isn't required here. So let's just use a different representation
for the cache key.
Closes https://github.com/astral-sh/uv/issues/4990.
This is pulled out of #4632 — a user noted that it would be useful to
use the `uv` crate from Rust. This makes it way easier to invoke `uv`
from Rust with arbitrary arguments as well as use various functionality
in the `uv` crate.
Note this is no longer needed for #4632 and is not particularly urgent.
Adds a `uv python pin` command to write to a `.python-version` file.
We support all of our Python version request formats. We also support a
`--resolved` flag to pin to a specific interpreter instead of the
provided version. We canonicalize the request with #4949, it's not just
printed verbatim. We always attempt to find the interpreter so we can
warn if it's not available. With `--resolved`, if we can't find the
interpreter we fail. If no arguments are provided, we'll attempt to
display the current pin.
In the future:
- We should confirm that this satisfies the `Requires-Python` metadata
if a `pyproject.toml` is present
- We should support writing to a `uv.python-version` field if
`pyproject.toml` or `uv.toml` are present
- We should support finding and updating the "nearest" Python version
file (looking in ancestors)
- We should support finding version files in workspaces
- We should support some sort of global pin
## Summary
Closes https://github.com/astral-sh/uv/issues/4671.
## Test Plan
```
❯ XDG_BIN_HOME="/Users/crmarsh/workspace/uv/foo bar" cargo run tool install black --force
Installed 2 executables: black, blackd
warning: `/Users/crmarsh/workspace/uv/foo bar` is not on your PATH. To use installed tools, run:
export PATH="/Users/crmarsh/workspace/uv/foo bar:$PATH"
```
Needed over in https://github.com/astral-sh/uv/pull/4674
These filters are relatively aggressive and may have a high false
positive rate, we don't need them for most tests so we leave them as
opt-in for now.
Special cases the `Any` request in output
e.g.,
```
❯ cargo run -q -- python install --isolated
warning: `uv python install` is experimental and may change without warning.
Searching for Python installations
Found existing installation: cpython-3.12.3-macos-aarch64-none
Python is already available. Use `uv python install <request>` to install a specific version.
```
instead of
```
❯ cargo run -q -- python install --isolated
warning: `uv python install` is experimental and may change without warning.
Searching for Python versions matching: any Python
Found existing installation for any Python: cpython-3.12.3-macos-aarch64-none
Python is already available. Use `uv python install <request>` to install a specific version.
```
## Summary
We currently store wheel URLs in an unparsed state because we don't have
a stable parsed representation to use with rykv. Unfortunately this
means we end up reparsing unnecessarily in a lot of places, especially
when constructing a `Lock`. This PR adds a `UrlString` type that lets us
avoid reparsing without losing the validity of the `Url`.
## Test Plan
Shaves off another ~10 ms from
https://github.com/astral-sh/uv/issues/4860.
```
➜ transformers hyperfine "../../uv/target/profiling/uv lock" "../../uv/target/profiling/baseline lock" --warmup 3
Benchmark 1: ../../uv/target/profiling/uv lock
Time (mean ± σ): 120.9 ms ± 2.5 ms [User: 126.0 ms, System: 80.6 ms]
Range (min … max): 116.8 ms … 125.7 ms 23 runs
Benchmark 2: ../../uv/target/profiling/baseline lock
Time (mean ± σ): 129.9 ms ± 4.2 ms [User: 127.1 ms, System: 86.1 ms]
Range (min … max): 123.4 ms … 141.2 ms 23 runs
Summary
../../uv/target/profiling/uv lock ran
1.07 ± 0.04 times faster than ../../uv/target/profiling/baseline lock
```
## Summary
Move completely off tokio's multi-threaded runtime. We've slowly been
making changes to be smarter about scheduling in various places instead
of depending on tokio's general purpose work-stealing, notably
https://github.com/astral-sh/uv/pull/3627 and
https://github.com/astral-sh/uv/pull/4004. We now no longer benefit from
the multi-threaded runtime, as we run on all I/O on the main thread.
There's one remaining instance of `block_in_place` that can be swapped
for `rayon::spawn`.
This change is a small performance improvement due to removing some
unnecessary overhead of the multi-threaded runtime (e.g. spawning
threads), but nothing major. It also removes some noise from profiles.
## Test Plan
```
Benchmark 1: ./target/profiling/uv (resolve-warm)
Time (mean ± σ): 14.9 ms ± 0.3 ms [User: 3.0 ms, System: 17.3 ms]
Range (min … max): 14.1 ms … 15.8 ms 169 runs
Benchmark 2: ./target/profiling/baseline (resolve-warm)
Time (mean ± σ): 16.1 ms ± 0.3 ms [User: 3.9 ms, System: 18.7 ms]
Range (min … max): 15.1 ms … 17.3 ms 162 runs
Summary
./target/profiling/uv (resolve-warm) ran
1.08 ± 0.03 times faster than ./target/profiling/baseline (resolve-warm)
```
## Summary
Avoid serializing and writing the lockfile if a cheap comparison shows
that the contents have not changed.
## Test Plan
Shaves ~10ms off of https://github.com/astral-sh/uv/issues/4860 for me.
```
➜ transformers hyperfine "../../uv/target/profiling/uv lock" "../../uv/target/profiling/baseline lock" --warmup 3
Benchmark 1: ../../uv/target/profiling/uv lock
Time (mean ± σ): 130.5 ms ± 2.5 ms [User: 130.3 ms, System: 85.0 ms]
Range (min … max): 126.8 ms … 136.9 ms 23 runs
Benchmark 2: ../../uv/target/profiling/baseline lock
Time (mean ± σ): 140.5 ms ± 5.0 ms [User: 142.8 ms, System: 85.5 ms]
Range (min … max): 133.2 ms … 153.3 ms 21 runs
Summary
../../uv/target/profiling/uv lock ran
1.08 ± 0.04 times faster than ../../uv/target/profiling/baseline lock
```
Adds a nice hint at the bottom of `uv help` output indicating how to get
more details about a specific command, roughly matching Cargo's
interface.
We use the short help and skip the pager for the root `uv help` since
it's intended to be a landing page for the help interface more than an
in-depth display. This also matches Cargo, though I like that they have
the global options above the commands and I've not changed that here.
## Summary
We now recreate the environment in `uv sync`, `uv tool install`, and `uv
tool run` if the underlying interpreter has been removed.
Closes https://github.com/astral-sh/uv/issues/4933.
This is an attempt to solve https://github.com/astral-sh/uv/issues/ by
applying the extra marker of the requirement to overrides and
constraints.
Say in `a` we have a requirements
```
b==1; python_version < "3.10"
c==1; extra == "feature"
```
and overrides
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10"
c==3; python_version >= "3.10"
```
Our current strategy is to discard the markers in the original
requirements. This means that on 3.12 for `a` we install `b==3`, but it
also means that we add `c` to `a` without `a[feature]`, causing #4826.
With this PR, the new requirement become,
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10" and extra == "feature"
c==3; python_version >= "3.10" and extra == "feature"
```
allowing to override markers while preserving optional dependencies as
such.
Fixes#4826
## Summary
Allows `--all` as an alternative to specifying specific targets.
## Test Plan
Verified that `cargo run python uninstall` still fails.
```
❯ cargo run python uninstall --all
Compiling uv-cli v0.0.1 (/Users/crmarsh/workspace/puffin/crates/uv-cli)
Compiling uv v0.2.23 (/Users/crmarsh/workspace/puffin/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.86s
Running `target/debug/uv python uninstall --all`
warning: `uv python uninstall` is experimental and may change without warning.
Searching for Python installations
Found existing installation: cpython-3.9.18-macos-aarch64-none
Found existing installation: cpython-3.8.18-macos-aarch64-none
Found existing installation: cpython-3.8.12-macos-aarch64-none
Found existing installation: cpython-3.12.1-macos-aarch64-none
Found existing installation: cpython-3.11.7-macos-aarch64-none
Found existing installation: cpython-3.10.13-macos-aarch64-none
Uninstalled cpython-3.10.13-macos-aarch64-none
Uninstalled cpython-3.11.7-macos-aarch64-none
Uninstalled cpython-3.12.1-macos-aarch64-none
Uninstalled cpython-3.8.12-macos-aarch64-none
Uninstalled cpython-3.8.18-macos-aarch64-none
Uninstalled cpython-3.9.18-macos-aarch64-none
Uninstalled 6 versions in 479ms
```
I feel like I'm always drowning in the help output from `uv` because we
have so many options.
I basically agree with the commentary in
https://github.com/clap-rs/clap/issues/4687 that having different
behaviors for `-h` and `--help` is surprising. I think `--help` is more
obvious for users and I want to optimize for that experience.
This roughly matches the help menus in Cargo and pip.
The `uv help` command can be used for long help. In #4906 and #4909 we
improve that command.
Extends #4904 which adds test cases for the existing behavior.
## Summary
In marker normalization, we now remove any markers that are redundant
with the `requires-python` specifier (i.e., always true for the given
Python requirement).
For example, given `iniconfig ; python_version >= '3.7'`, we can remove
the `python_version >= '3.7'` marker when resolving with
`--python-version 3.8`.
Closes#4852.
## Summary
Given `python_version != '3.8' and python_version < '3.10'`, the first
term was expanded to `python_version < '3.8'` and `python_version >
'3.8'`. We then AND'd all three terms together. We don't seem to have a
way to differentiate between the terms to AND and the terms to OR in the
normalization code (it all gets flattened together), so instead this PR
expands the expressions at the leaf level and then flattens them at the
level above when appropriate.
Closes https://github.com/astral-sh/uv/issues/4910.
## Summary
Closes https://github.com/astral-sh/uv/issues/4915.
```
error: No interpreter found for Python 3.12.4 in virtual environments, managed installations, or system path
```
## Summary
Closes https://github.com/astral-sh/uv/issues/4848.
## Test Plan
```
> cargo run -- run -vv --preview --isolated --python 3.12.4 python -V
error: No interpreter found for Python 3.12.4 in virtual environments or managed installations or system path
```
## Summary
More marker simplification:
- Filters out redundant subtrees based on outer expressions, e.g. `a and (a or
b)` simplifies to `a`.
- Flattens nested trees internally, e.g. `(a and b) and c`
Resolves https://github.com/astral-sh/uv/issues/4536.
The PR #4707 introduced the notion of "version narrowing," where a
Requires-Python constraint was _possibly_ narrowed whenever the
universal resolver created a fork. The version narrowing would occur
when the fork was a result of a marker expression on `python_version`
that is *stricter* than the configured `Requires-Python` (via, say,
`pyproject.toml`).
The crucial conceptual change made by #4707 is therefore that
`Requires-Python` is no longer an invariant configuration of resolution,
but rather a mutable constraint that can vary from fork to fork. This in
turn can result in some cases, such as in #4885, where different
versions of dependencies are selected. We aren't sure whether we can fix
those or not, with version narrowing, so for now, we do this revert to
restore the previous behavior and we'll try to address the version
narrowing some other time.
This also adds the case from #4885 as a regression test, ensuring that
we don't break that in the future. I confirmed that with version
narrowing, this test outputs duplicate distributions. Without narrowing,
there are no duplicates.
Ref #4707, Fixes#4885
This test was, I believe, relying on the XDG_DATA_HOME environment
variable not being set. When it is set, as is the case in my
environment, `uv tool run` will respect it and install `black` for this
particular test into my actual XDG_DATA_HOME directory. We fix this by
setting `XDG_DATA_HOME` explicitly.
By using `Box::pin(run())` we can reduce the artificial stack size for
running tests on windows in debug mode from 8MB to 2MB. I've checked and
1MB/no custom stack size still fail tests, e.g.
`add_workspace_editable`.
This test is failing most times for me when running nextest locally,
failing the overall test run, so i'm deactivating it for now. I'm still
not sure what the root cause here is. It seems to have something to do
with python stdin not being ready immediately after we spawn the process
and us being too fast.
## Summary
Resolves#4834
## Test Plan
```sh
# 3.12.3 is a `install_only` archive
$ cargo run -- python install --preview --force 3.12.3
# 3.9.4 has only `full` archive
$ cargo run -- python install --preview --force 3.9.4
```
## Summary
Partially closes#1917
This PR picks up on some of the great work from #1864 and opted to keep
`panic_immediate_abort` (for size reasons). I split the PR in different
isolated commits in case we want to separate/cherry-pick them out.
1. The first commit ports mostly all std changes from that PR into this
PR. Binary sizes stayed the same ~16kb.
2. The second commit migrates our existing usage of windows-sys to
windows for a safer ffi calls with Results!. It also changes all large
unsafe blocks to be isolated to the actual unsafe calls, and switches
some areas to use std such as getenv port ( which seemed buggy! ) from
launcher.c. In addition, this also adds more error checking in order to
match some missing assertions from distlib's launcher.c. Note, due to
the additional .text data, the binary sizes increased to ~20.5kb, but we
can cut back on some of the added error msgs as needed.
3. The third commit switches to using xwin for building on all 3
supported trampoline targets for sanity, and adds a CI bloat check for
core::fmt and panic as a precaution. Sadly, this will invalidate the
xwin cache on the first run.
## Test Plan
Most changes were tested on a couple of local GUI apps and console apps,
also tested some of the error states manually by using SetLastError at
different points in the code and/or passing in invalid handles.
I'm not sure how far we can get with migrating some of the other calls
without increasing binary size substantially. An initial attempt at
using std::path didn't seem so bad size wise when I tried it (~1k). On
other cases, such as std::process::exit added ~10k to the total binary
size.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
If you pass `--isolated` but no `--with`, at present, we don't create
any environment (so `--python` isn't respected and `python` will fail
entirely if it wasn't already in your path). Now, we create a base
environment in `--isolated` even if `with` wasn't provided.
Closes https://github.com/astral-sh/uv/issues/4846.
Closes https://github.com/astral-sh/uv/issues/4776.
## Summary
Like https://github.com/astral-sh/uv/pull/4808 but with a few more
changes. I suspect this will require some bikeshedding but I find the
use of "installation" and "installed" in the same sentence to be kind of
a lot.
## Summary
There are a few ideas at play here:
1. pip always strips versions to the release when evaluating against a
`Requires-Python`, so we now do the same. That means, e.g., using
`3.13.0b0` will be accepted by a project with `Requires-Python: >=
3.13`, which does _not_ adhere to PEP 440 semantics but is somewhat
intuitive.
2. Because we know we'll only be evaluating against release-only
versions, we can use different semantics in PubGrub that let us collapse
ranges. For example, `python_version >= '3.10' or python_version <
'3.10'` can be collapsed to the truthy marker.
Closes https://github.com/astral-sh/uv/issues/4714.
Closes https://github.com/astral-sh/uv/issues/4272.
Closes https://github.com/astral-sh/uv/issues/4719.
Remove wheels from the lockfile that don't match the required python
version. For example, we remove
`charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl` when we have
`requires-python = ">=3.12"`.
Our snapshots barely show changes since we avoid the large binaries for
which matters. Here are 3 real world `uv.lock` before/after comparisons
to show the large difference:
*
[warehouse](https://gist.github.com/konstin/9a1ed6a32b410e250fcf4c6ea8c536a5)
(5677 -> 4214)
*
[transformers](https://gist.github.com/konstin/5636281b5226f64aa44ce3244d5230cd)
(6484 -> 5816)
*
[github-wikidata-bot](https://gist.github.com/konstin/ebbd7b9474523aaa61d9a8945bc02071)
(793 -> 454)
We only remove wheels we are certain don't match the python version and
still keep those with unknown tags. We could remove even more wheels by
also considering other markers, e.g. removing linux wheels for a
windows-only dep, but we would trade complex, easy-to-get-wrong logic
for diminishing returns.
## Summary
Check the sha256 checksum when downloading a managed python toolchain.
## Test Plan
```sh
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12.3 (any-3.12.3-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
Installed Python 3.12.3 to C:\Users\jo\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none
Installed 1 installation in 6s
$ cargo run -- python uninstall 3.12
$ # manually change the hash in `crates/uv-python/src/downloads.inc`
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12 (any-3.12-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
error: Hash mismatch for `cpython-3.12.3-windows-x86_64-none`
Expected:
xx
Computed:
776568c92c5f3b47dbf5f17c1c58578f70d75a32654419a158aa8bdc6f95b09a
```
## Summary
This seems like another good candidate for environment caching. If you
run a script repeatedly, we can just use the existing cached
environment.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Addresses https://github.com/astral-sh/uv/issues/4330, to reduce
duplication in the client creation logic.
## Test Plan
<!-- How was it tested? -->
https://github.com/astral-sh/uv/pull/4729#issuecomment-2204681655
## Summary
The resolver sometimes starts HTTP requests that end up not being
necessary. When dropping the Tokio runtime before exiting we currently
wait for those to complete. This can cause noticeable hangs in the CLI,
particularly when the runtime is blocked on slow DNS resolution.
Resolves https://github.com/astral-sh/uv/issues/4599.
## Test Plan
This change resolves any reproducible hangs for me locally.
## Summary
The basic strategy:
- When the user does `uv tool run`, we resolve the `from` and `with`
requirements (always).
- After resolving, we generate a hash of the requirements. For now, I'm
just converting to a lockfile and hashing _that_, but that's an
implementation detail.
- Once we have a hash, we _also_ hash the interpreter.
- We then store environments in
`${CACHE_DIR}/${INTERPRETER_HASH}/${RESOLUTION_HASH}`.
Some consequences:
- We cache based on the interpreter, so if you request a different
Python, we'll create a new environment (even if they're compatible).
This has the nice side-effect of ensuring that we don't use environments
for interpreters that were later deleted.
- We cache the `from` and `with` together. In practice, we may want to
cache them separately, then layer them? But this is also an
implementation detail that we could change later.
- Because we use the lockfile as the cache key, we will invalidate the
cache when the format changes. That seems ok, but we could improve it in
the future by generating a stable hash from a lockfile that's
independent of the schema.
Closes https://github.com/astral-sh/uv/issues/4752.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes#4653
## Summary
Adds the tool version to the list command right beside the tool name
```
$ uv tool list
black v24.2.0
```
Following the proposed format discussed in #4653
## Test Plan
`cargo test tool_list`
<!-- How was it tested? -->
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
The changes in https://github.com/astral-sh/uv/pull/4708 caused an
overflow in debug mode only of the 1MB default stack size in windows
during clap. This means that even trivial wrong argument tests would
fail without increasing the stack size. As remedy, we box the clap
types.
This fell out of my investigation of
https://github.com/astral-sh/uv/issues/4774 but the bug was fixed by the
reporter in #4775
- Adds support for `GH_TOKEN` authentication again — basically needed to
avoid rate limits when hacking on this.
- Clarifies some handling and logging of flavors
## Summary
This doesn't cache the tool environment; rather, it just uses the `tool
install` environment if it satisfies the request.
Closes https://github.com/astral-sh/uv/issues/4742.
Previously this displayed:
```
❯ cargo run -q -- --help
The command line interface for the uv binary.
Usage: uv [OPTIONS] <COMMAND>
```
This is.. weird. Here I remove it entirely. I could see adding
`about_long` text being helpful in the future.
Fix#4774.
## Summary
Change the python interpreter for linux installed with `uv python` to an
optimized one.
## Test Plan
I ran the following command on Linux (glibc) to confirm that an
optimized (not debug built) Python is installed.
```bash
# install python
uv python install 3.12.3
# check build type
uv run python -c "import sysconfig;print(sysconfig.get_config_var('Py_DEBUG'))"
0
```
## Summary
This used to be necessary because we purged the cache in the
`InstallPlan` if the user passed `--reinstall`. _However_, we later
changed the cache to be append-only.
## Test Plan
I ran through the test plan in https://github.com/astral-sh/uv/pull/933,
which includes an integration test and running `uv pip install
--reinstall` with:
```text
setuptools
devpi @ e334eb4dc9bb023329e4b610e4515b/devpi-2.2.0.tar.gz
```
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes#4654
## Summary
The purpose of this is to show the entrypoints of each tool when running
`uv tool list` as below:
```
$ uv tool list
black
black
blackd
```
I used the proposed formatting as it was written in #4653 by @blueraft.
I had to use spaces instead of tabs in order to make the test
successful. Indeed in the test we are using a raw string and I did not
manage to make the test pass when escaping the tab in the list.rs file
so I used spaces everywhere.
I had a deeper look into #4653 as well but it is more difficult as we
need to get the version of the tool in the Tool object, I will continue
on this next one later.
Please tell me if anything else is needed I tried to follow the
contribution guidelines but I might have forgotten something.
Have a great day!
## Test Plan
`cargo clippy`
then by using the local version of uv as described in the Readme.md.
```
my-computer :~/mypath/uv$ cargo run -- tool list
Compiling uv-cli v0.0.1 (/mypath/uv/crates/uv-cli)
Compiling uv v0.2.18 (/mypath/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.69s
Running `target/debug/uv tool list`
warning: `uv tool list` is experimental and may change without warning.
black
black
blackd
isort
isort
isort-identify-imports
```
and
`cargo test tool_list`
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
For now the semantics are such that if the requested requirements from
the command line don't match the receipt (or if any `--reinstall` or
`--upgrade` is requested), we proceed with an install, passing the
`--reinstall` and `--upgrade` to the underlying Python environment.
This may lead to some unintuitive behaviors, but it's simplest for now.
For example:
- `uv tool install black<24` followed by `uv tool install black
--upgrade` will install the latest version of `black`, removing the
`<24` constraint.
- `uv tool install black --with black-plugin` followed by `uv tool
install black` will remove `black-plugin`.
Closes https://github.com/astral-sh/uv/issues/4659.
In #3514 and #2755, users had intermittent network errors, but it was
not always clear whether we had already retried these requests or not.
Building upon https://github.com/TrueLayer/reqwest-middleware/pull/159,
this PR adds the number of retries to the error message, so we can see
at first glance where we're missing retries and where we might need to
change retry settings.
Example error trace:
```
Could not connect, are you offline?
Caused by: Request failed after 3 retries
Caused by: error sending request for url (https://pypi.org/simple/uv/)
Caused by: client error (Connect)
Caused by: dns error: failed to lookup address information: Name or service not known
Caused by: failed to lookup address information: Name or service not known
```
This code is ugly since i'm missing a better pattern for attaching
context to reqwest middleware errors in
https://github.com/TrueLayer/reqwest-middleware/pull/159.
## Summary
Given:
```text
numpy >=1.26 ; python_version >= '3.9'
numpy <1.26 ; python_version < '3.9'
```
When resolving for Python 3.8, we need to narrow the `requires-python`
requirement in the top branch of the fork, because `numpy >=1.26` all
require Python 3.9 or later -- but we know (in that branch) that we only
need to _solve_ for Python 3.9 or later.
Closes https://github.com/astral-sh/uv/issues/4669.
## Summary
This is required to solve https://github.com/astral-sh/uv/issues/4669,
because the `Requires-Python` version can now vary across a resolution.
For example, within certain forks, we might have a more narrow range,
which would allow us to use distributions that would not be allowed for
the global resolution.
This should be fine because `requires-python` is part of the package
metadata, so it should be consistent between files within a package
version. As such, there shouldn't be any risk that we incorrectly
prioritize distributions by omitting this information.
(To be more specific, the risk is something like: we prioritize some
wheel over a source distribution within a package-version, so we don't
track the source distribution at all. Then, later, when we choose a
candidate, we see that the wheel doesn't meet the `Requires-Python`
requirement, even though the source distribution _would've_ met it. If
files within a distribution could have varied support, this would be a
real risk.)
I think `--toolchain-preference system` is sufficiently clear and
`--toolchain-preference prefer-system` is excessively verbose. This was
discussed in the original pull request at
https://github.com/astral-sh/uv/pull/4424 but because we had a case for
preferring "installed managed" toolchains I was hesitant to change it.
Now that I've dropped that in #4601, I think we can drop the prefix.
Adds a `toolchain-fetch` option alongside `toolchain-preference` with
`automatic` (default) and `manual` values allowing automatic toolchain
fetches to be disabled (replaces
https://github.com/astral-sh/uv/pull/4425). When `manual`, toolchains
must be installed with `uv toolchain install`.
Note this was previously implemented with `if-necessary`, `always`,
`never` variants but the interaction between this and
`toolchain-preference` was too confusing. By reducing to a binary
option, things should be clearer. The `if-necessary` behavior moved to
`toolchain-preference=installed`. See
https://github.com/astral-sh/uv/pull/4601#discussion_r1657839633 and
https://github.com/astral-sh/uv/pull/4601#discussion_r1658658755
Closes https://github.com/astral-sh/uv/issues/4476
Originally, this used the changes in #4642 to invoke `main()` from a
`uvx` binary. This had the benefit of `uvx` being entirely standalone at
the cost of doubling our artifact size. We think that's the incorrect
trade-off.
Instead, we assume `uvx` is always next to `uv` and create a tiny binary
(<1MB) that invokes `uv` in a child process. This seems preferable to a
`cargo-dist` alias because we have more control over it. This binary
should "just work" for all of our cargo-dist distributions and
installers, but we'll need to add a new entry point for our PyPI
distribution. I'll probably tackle support there separately?
```
❯ ls -lah target/release/uv target/release/uvx
-rwxr-xr-x 1 zb staff 31M Jun 28 23:23 target/release/uv
-rwxr-xr-x 1 zb staff 452K Jun 28 23:22 target/release/uvx
```
This includes some small overhead:
```
❯ hyperfine --shell=none --warmup=100 './target/release/uv tool run --help' './target/release/uvx --help' --min-runs 2000
Benchmark 1: ./target/release/uv tool run --help
Time (mean ± σ): 2.2 ms ± 0.1 ms [User: 1.3 ms, System: 0.5 ms]
Range (min … max): 2.0 ms … 4.0 ms 2000 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: ./target/release/uvx --help
Time (mean ± σ): 2.9 ms ± 0.1 ms [User: 1.7 ms, System: 0.9 ms]
Range (min … max): 2.8 ms … 4.2 ms 2000 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
./target/release/uv tool run --help ran
1.35 ± 0.09 times faster than ./target/release/uvx --help
```
I presume there may be some other downsides to a child process? The
wrapper is a little awkward. We could consider `execv` but this is
complicated across platforms. An example implementation of that over in
[monotrail](433af5aed9/crates/monotrail/src/monotrail.rs (L764-L799)).
## Summary
I ended up needing this for https://github.com/astral-sh/uv/issues/4664
but I think it's a good change more broadly. We should be able to share
this cached information across operations within a given invocation.
## Summary
These are changing in one of my branches but I can't tell _what's_
changing. Some tests include the lock, but others don't. This PR adds it
for all successful resolves in the suite.
## Summary
You can now add `managed = false` under `[tool.uv]` in a
`pyproject.toml` to explicitly opt out of the project and workspace
APIs.
If a project sets `managed = false`, we will (1) _not_ discover it as a
workspace root, and (2) _not_ discover it as a workspace member (similar
to using `exclude` in the workspace parent).
Closes https://github.com/astral-sh/uv/issues/4551.
## Summary
This doesn't actually change any behaviors, but it does make it a bit
easier to solve #4669, because we don't have to support "version
narrowing" for the non-`RequiresPython` variants in here. Right now, the
semantics are kind of muddied, because the `target` variant is
_sometimes_ interpreted as an exact version and sometimes as a lower
bound.
## Summary
`GitDatabase::contains` previously only parsed the commit to see if it
was a valid hash and didn't verify if the commit existed in the object
database. This led to the database never being updated.
Resolves https://github.com/astral-sh/uv/issues/4378.
## Test Plan
Added a test that fails without this change.
It's hard to talk about solve state and resolver state, so i'm renaming
them to fork state and resolver state, indicating the hierarchy between
more directly.
## Summary
Closes https://github.com/astral-sh/uv/issues/4688.
## Test Plan
```
❯ cargo run tool install ruff
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv tool install ruff`
warning: `uv tool install` is experimental and may change without warning.
Resolved 1 package in 136ms
Installed 1 package in 3ms
+ ruff==0.5.0
No entrypoints to install for tool `ruff`
```
## Summary
Packages that provide scripts that _aren't_ Python entrypoints need to
respected in `uv tool install`. For example, Ruff ships a script in
`ruff-0.5.0.data/scripts`.
Unfortunately, the `.data` directory doesn't exist in the virtual
environment at all (it's removed, per the spec, after install). So this
PR changes the entry point detection to look at the `RECORD` file, which
is the only evidence that the scripts were installed.
Closes https://github.com/astral-sh/uv/issues/4691.
## Test Plan
`cargo run uv tool install ruff` (snapshot tests to-come)
## Summary
Resolves#4483Resolves#4484
## Test Plan
`cargo test`
```sh
❯ cargo run -- toolchain dir
warning: `uv toolchain dir` is experimental and may change without warning.
/Users/ahmedilyas/Library/Application Support/uv/toolchains
❯ cargo run -- tool dir
warning: `uv tool dir` is experimental and may change without warning.
/Users/ahmedilyas/Library/Application Support/uv/tools
```
## Summary
I think this may have just been a typo.
Closes https://github.com/astral-sh/uv/issues/4692.
## Test Plan
Run `cargo run tool install flask --force --reinstall` repeatedly.
## Summary
I noticed that `init_environment` and `find_interpreter` were both
calling `find_environment`, which seemed like a code smell to me.
Instead, `find_interpreter` now returns either a compatible environment
or an interpreter (if no compatible environment was found).
Additionally, `interpreter_meets_requirements` now no longer validates
`requires-python` if `--python` or `.python-version` is set. Instead, we
warn, which matches the behavior we get when creating a new environment
at the bottom of `find_interpreter`.
In total, I think this makes the data flow in project interpreter
discovery less repetitive and easier to reason about.
## Summary
This should both make it faster to solve forks (since we have a guess
for a valid resolution, and will bias towards packages we've already
fetched) and improve consistency between forks.
Closes https://github.com/astral-sh/uv/issues/4617.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
resolves https://github.com/astral-sh/uv/issues/4651
(pruning needs to happen at the parent level so that the number of
children being used to figure out the output is correct)
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
added a test that would've caught this bug 🌵
<!-- How was it tested? -->
## Summary
`remove_edge` will invalidate the last index in the graph, so we need to
ensure that each index we look at is "earlier" than the last.
Co-authored-by: bluss <bluss@users.noreply.github.com>
## Summary
When a constraint is applied to a requirement with a marker, the marker
needs to be propagated to the constraint.
If both the constraint and the requirement have a marker, they need to
be merged together (via `and`).
Closes https://github.com/astral-sh/uv/issues/4575.
## Summary
This PR dodges some of the bigger issues raised by
https://github.com/astral-sh/uv/pull/4554 and
https://github.com/astral-sh/uv/pull/4555 by _not_ changing any of the
bigger semantics around syncing and instead merely changing virtual
workspace roots to sync all packages in the workspace (rather than
erroring due to being unable to find a project).
Closes#4541.
We need this to power uninstallations!
The latter two commits were reviewed in:
- #4637
- #4638
Note this is a breaking change for existing tool installations, but it's
in preview and very new. In the future, we'll need a clear upgrade path
for tool receipt changes.
This centralizes writing out the DistributionId as TOML. This is again
just a refactor. No behavioral changes were made. In a subsequent
commit, we will tweak how `source` is written.
Looks much better than #4618:
```
DEBUG Pre-fork split universal took 0.644s
DEBUG Split python_version >= '3.12' and platform_machine == 'aarch64' and platform_system == 'Darwin' and platform_system == 'Linux' took 0.659s
DEBUG Split python_version == '3.9' and platform_machine == 'arm64' and platform_system == 'Darwin' took 0.291s
```
The journey here can be seen in:
- #4587
- #4589
- #4594
I collapsed all the commits here because only the last one in the stack
got us to a "correct" error message.
There are a few architectural changes:
- We have a dedicated `MissingEnvironment` and `EnvironmentNotFound`
type for `PythonEnvironment::find` allowing different error messages
when searching for environments
- `ToolchainNotFound` becomes a struct with the `ToolchainRequest` which
greatly simplifies missing toolchain error formatting
- `ToolchainNotFound` tracks the `EnvironmentPreference` so it can
accurately report the locations checked
The messages look like this now, instead of the bland (and often
incorrect): "No Python interpreter found in system toolchains".
```
❯ cargo run -q -- pip sync requirements.txt
error: No virtual environment found
❯ UV_TEST_PYTHON_PATH="" cargo run -q -- pip sync requirements.txt --system
error: No system environment found
❯ UV_TEST_PYTHON_PATH="" cargo run -q -- pip sync requirements.txt --python 3.12
error: No virtual environment found for Python 3.12
❯ UV_TEST_PYTHON_PATH="" cargo run -q -- pip sync requirements.txt --python 3.12 --system
error: No system environment found for Python 3.12
❯ UV_TEST_PYTHON_PATH="" cargo run -q -- toolchain find 3.12 --preview
error: No toolchain found for Python 3.12 in system path
❯ UV_TEST_PYTHON_PATH="" cargo run -q -- pip compile requirements.in
error: No toolchain found in virtual environments or system path
```
I'd like to follow this with hints, suggesting creating an environment
or using system in some cases.
This includes a functional change, we now skip the forked state pop/push
if we didn't fork.
From transformers:
```
DEBUG Pre-fork split universal took 0.036s
DEBUG Split python_version >= '3.10' and python_version >= '3.10' and platform_system == 'Darwin' and python_version >= '3.11' and python_version >= '3.12' and python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64' took 0.048s
DEBUG Split python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64' and python_version >= '3.7' and python_version >= '3.8' and python_version >= '3.9' took 0.038s
```
The messages could use simplification from
https://github.com/astral-sh/uv/issues/4536
We can consider nested spans in the future but this works nicely for
now.
https://github.com/astral-sh/uv/pull/2419 appears to have only applied
this retry to wheels that were already downloaded (though I would have
to look more carefully to be certain). In
https://github.com/astral-sh/uv/issues/1491, we've gotten continued
reports of spurious failures on Windows and tracing reveals that we are
not applying our retry logic during the rename. I believe we're in this
code path — switching to our backoff retry should resolve the failures.
## Summary
resolves https://github.com/astral-sh/uv/issues/4609
previously, the implementation of `required_with_no_extra` was
incorrect, particularly when there are packages that do not require any
extras but have other types of markers.
## Test Plan
the existing tests also did cover this (my bad... missed it) but added a
smaller test since this bug would've been more obvious with this new
test.
## Summary
It turns out that `Topo` only works on graphs without cycles. If a graph
has a cycle, it seems to bail early. So we were losing markers for trees
that contain cycles (like Poetry, which depends on
`poetry-plugin-export`, which depends on Poetry).
Now, we remove cycles beforehand and re-add those edges afterwards.
It's a bit hard for me to reason about the implications of this. The way
that marker propagation works is that we do visit the nodes in-order and
propagate the markers from any incoming to any outgoing edges. We only
do this at a single depth (rather than recursively) because we visit the
nodes in-order anyway. But if you have a cycle... then in theory you
might need to propagate the markers recursively? Or maybe not?
As an example:
`A -> B -> C -> D -> B`
If `A -> B` has `sys_platform == 'darwin'`, and then `D -> B` has
`python_version >= '3.7`... then we don't need to propagate
`python_version >= '3.7'` back to `B` or any of its dependencies,
because the condition would be `(sys_platform == 'darwin' or
python_version >= '3.7) or sys_platform == 'darwin'`, which is
equivalent to `sys_platform == 'darwin'`.
Closes#4584.
This PR contains two style changes to the lockfile:
* Always indent lists of objects, even with they are only a single
element.
* Use 4 spaces instead of tabs for indenting, to mirror what we do in
the ruff formatter.
## Summary
Open to just making this a warning but no strong opinion.
Closes https://github.com/astral-sh/uv/issues/4593.
## Test Plan
Failure:
```
❯ echo "pandas==2.2.2" | cargo run pip compile --universal -p 3.11 --no-header - --python-platform linux
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
Running `target/debug/uv pip compile --universal -p 3.11 --no-header - --python-platform linux`
error: the argument '--universal' cannot be used with '--python-platform <PYTHON_PLATFORM>'
Usage: uv pip compile --universal --python-version <PYTHON_VERSION> --no-header <SRC_FILE>...
For more information, try '--help'.
```
Use indented inline tables for `distribution.dependencies`,
`distribution.optional-dependencies` and
`distribution.dev-dependencies`.
The new style is more concise (see examples below) and it makes the
association between a distribution and its dependencies clearer
(previously, they were both individual `[[...]]` blocks separated by
newlines). The style is optimized for small, meaningful diffs by placing
each dependency on a single line with a final trailing comma. Whenever a
dependency is added, removed or changed, there should be a one line diff
in `distribution.dependencies`. The final trailing comma ensures that
adding a dependency doesn't change the line ahead.
Part of #3611
## Examples
### Simple workspace package
Before:
```toml
[[distribution]]
name = "bird-feeder"
version = "1.0.0"
source = "editable+packages/bird-feeder"
[[distribution.dependencies]]
name = "anyio"
[[distribution.dependencies]]
name = "seeds"
```
After:
```toml
[[distribution]]
name = "bird-feeder"
version = "1.0.0"
source = "editable+packages/bird-feeder"
dependencies = [
{ name = "anyio" },
{ name = "seeds" },
]
```
### Flask
Before:
```toml
[[distribution]]
name = "flask"
version = "3.0.2"
source = "registry+https://pypi.org/simple"
sdist = { url = "a89e8120fa0bbafcb2c2387c0317be/flask-3.0.2.tar.gz", hash = "sha256:822c03f4b799204250a7ee84b1eddc40665395333973dfb9deebfe425fefcb7d", size = 675248 }
wheels = [{ url = "aa98bfe0ebf27ce224fb4f766acb23/flask-3.0.2-py3-none-any.whl", hash = "sha256:3232e0e9c850d781933cf0207523d1ece087eb8d87b23777ae38456e2fbe7c6e", size = 101300 }]
[[distribution.dependencies]]
name = "blinker"
[[distribution.dependencies]]
name = "click"
[[distribution.dependencies]]
name = "itsdangerous"
[[distribution.dependencies]]
name = "jinja2"
[[distribution.dependencies]]
name = "werkzeug"
[distribution.optional-dependencies]
[[distribution.optional-dependencies.dotenv]]
name = "python-dotenv"
```
After:
```toml
[[distribution]]
name = "flask"
version = "3.0.2"
source = "registry+https://pypi.org/simple"
sdist = { url = "a89e8120fa0bbafcb2c2387c0317be/flask-3.0.2.tar.gz", hash = "sha256:822c03f4b799204250a7ee84b1eddc40665395333973dfb9deebfe425fefcb7d", size = 675248 }
dependencies = [
{ name = "blinker" },
{ name = "click" },
{ name = "itsdangerous" },
{ name = "jinja2" },
{ name = "werkzeug" },
]
wheels = [{ url = "aa98bfe0ebf27ce224fb4f766acb23/flask-3.0.2-py3-none-any.whl", hash = "sha256:3232e0e9c850d781933cf0207523d1ece087eb8d87b23777ae38456e2fbe7c6e", size = 101300 }]
[distribution.optional-dependencies]
dotenv = [
{ name = "python-dotenv" },
]
```
### Forking
Before:
```toml
[[distribution]]
name = "project"
version = "0.1.0"
source = "editable+."
[[distribution.dependencies]]
name = "package-a"
version = "4.3.0"
source = "registry+https://astral-sh.github.io/packse/0.3.29/simple-html/"
marker = "sys_platform == 'darwin'"
[[distribution.dependencies]]
name = "package-a"
version = "4.4.0"
source = "registry+https://astral-sh.github.io/packse/0.3.29/simple-html/"
marker = "sys_platform == 'linux'"
[[distribution.dependencies]]
name = "package-b"
marker = "sys_platform == 'linux'"
[[distribution.dependencies]]
name = "package-c"
marker = "sys_platform == 'darwin'"
```
After:
```toml
[[distribution]]
name = "project"
version = "0.1.0"
source = "editable+."
dependencies = [
{ name = "package-a", version = "4.3.0", source = "registry+https://astral-sh.github.io/packse/0.3.29/simple-html/", marker = "sys_platform == 'darwin'" },
{ name = "package-a", version = "4.4.0", source = "registry+https://astral-sh.github.io/packse/0.3.29/simple-html/", marker = "sys_platform == 'linux'" },
{ name = "package-b", marker = "sys_platform == 'linux'" },
{ name = "package-c", marker = "sys_platform == 'darwin'" },
]
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes#1329.
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Mentions use of seed packages during `uv venv --seed`, and clarifies the
divergence in behavior when using Python 3.12+.
## Test Plan
<!-- How was it tested? -->
`cargo nextest run --test venv`
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Moves `--from` to a hidden argument — we allow it still but we validate
that it is compatible with whatever is passed to `uv tool install
<package>`. The positional package can now be a full specification,
allowing things like `uv tool install black==24.2.0`.
## Summary
In the dependency refactor (https://github.com/astral-sh/uv/pull/4430),
the logic for requirements and constraints was combined. Specifically,
we were applying constraints _before_ filtering on markers and extras,
and then applying that same filtering to the constraints. As a result,
constraints that should only be activated when an extra is enabled were
being enabled unconditionally.
Closes https://github.com/astral-sh/uv/issues/4569.
## Summary
- Adds a `--extra` flag to `uv add` that allows activating extras
without the PEP508 syntax.
- `uv add` now errors if the update is ambiguous (e.g. the dependency is
present twice with different markers)
- `uv add` is smarter about updates. For example, `uv add flask==3.0.0`
followed by `uv add flask --extra dotenv` preserves the previous version
specifier.
Resolves https://github.com/astral-sh/uv/issues/4419.
Adds support for `--reinstall` and `--reinstall-package` to `uv tool
install`. These are already available via the installer settings, we
just respect them now.
`--reinstall` implies a recreation of the environment and reinstallation
of the entry points.
`--reinstall-package` will only update a subset of the environment. If
the target package is the one with the entry points, we'll reinstall the
entry points. Otherwise, the entry points are not changed.
Adds detection of existing entry points, avoiding clobbering entry
points that were installed by another tool. If we see any existing entry
point collisions, we'll stop instead of overwriting them. The `--force`
flag can be used to opt-in to overwriting the files; we can't use `-f`
because it's taken by `--find-links` which is silly. The `--force` flag
also implies replacing a tool previously installed by uv (the
environment is rebuilt).
Similarly, #4504 adds support for reinstalls that _will not_ clobber
entry points managed by other tools.
## Summary
If the package _isn't_ marked as `workspace = true`, locking will fail
given:
```rust
let workspace_package_declared =
// We require that when you use a package that's part of the workspace, ...
!workspace.packages().contains_key(&requirement.name)
// ... it must be declared as a workspace dependency (`workspace = true`), ...
|| matches!(
source,
Some(Source::Workspace {
// By using toml, we technically support `workspace = false`.
workspace: true,
..
})
)
// ... except for recursive self-inclusion (extras that activate other extras), e.g.
// `framework[machine_learning]` depends on `framework[cuda]`.
|| &requirement.name == project_name;
if !workspace_package_declared {
return Err(LoweringError::UndeclaredWorkspacePackage);
}
```
Closes https://github.com/astral-sh/uv/issues/4552.
## Summary
This PR modifies `uv run` to fallback to discovering an interpreter
(e.g., a local `.venv`) if the command is run outside of a workspace.
`uv run --isolated` continues to completely skip workspace _and_
interpreter discovering, only installing whatever's provided with
`--with`.
The next step here is adding some ergonomic controls for enabling this
behavior even if your project is technically in a workspace (i.e., you
have a `pyproject.toml` but aren't using the Project APIs and don't want
locking etc.). I could imagine a setting in `pyproject.toml` that's also
exposed on the command-line. Something like: `managed = false` or
`project = false`.
See: https://github.com/astral-sh/uv/issues/3836.
This is the minimal "working" implementation. In summary, we:
- Resolve the requested requirements
- Create an environment at `$UV_STATE_DIR/tools/$name`
- Inspect the `dist-info` for the main requirement to determine its
entry points scripts
- Link the entry points from a user-executable directory
(`$XDG_BIN_HOME`) to the environment bin
- Create an entry at `$UV_STATE_DIR/tools/tools.toml` tracking the
user's request
The idea with `tools.toml` is that it allows us to perform upgrades and
syncs, retaining the original user request (similar to declarations in a
`pyproject.toml`). I imagine using a similar schema in the
`pyproject.toml` in the future if/when we add project-levle tools. I'm
also considering exposing `tools.toml` in the standard uv configuration
directory instead of the state directory, but it seems nice to tuck it
away for now while we iterate on it. Installing a tool won't perform a
sync of other tool environments, we'll probably have an explicit `uv
tool sync` command for that?
I've split out todos into follow-up pull requests:
- #4509 (failing on Windows)
- #4501
- #4504Closes#4485
`ResolverState::choose_version` had become huge, with an odd match due
to the url handling from #4435. This refactoring breaks it into
`choose_version`, `choose_version_registry` and `choose_version_url`. No
functional changes.
In the case of a direct URL sdist, it includes a hash, and this hash is
not (and probably should not) be part of the `source`. The URL is part
of the source because it permits uniquely identifying this particular
package as distinct from any other package with the same name. But, we
should still include the hash.
So in this commit, we rejigger what we did previously to make it so the
`SourceDist` value isn't even constructed at all when it isn't needed.
This also in turn lets us make the hash field required (which we will do
in a subsequent commit).
This does mean the URL is stored twice for direct URL dependencies in
the lock file. This seems non-ideal. We could make the URL for the sdist
optional, but this seems like a bridge too far? Another choice is to add
a new key to `distribution` that is just `direct-url-hash`, but that
also seems mucky.
Maybe the duplication here is okay given the relative rarity of direct
URL dependencies.
This updates all of the test snapshots where `sdist` was
strictly redundant and could be removed.
Note that there is one test failure whose snapshot I didn't
update: one where there is a direct URL dependency. In this
case, the sdist entry isn't strictly redundant, as it includes
a hash that isn't present in the source. We'll deal with that
in a subsequent commit.
This fixes an issue in the lock file where, in cases where we had a
non-registry sdist, the information in the sdist was strictly redundant
with the information in the source. This was born out in the code
already where the `sdist` field was only ever used to build a source
distribution type when the source was a registry. In all other cases,
the source distribution data can be materialized from the `source`
field.
This makes it clear that an actual `sdist` is only required when a
distribution is from a registry. In all other cases, a source
distribution is manufactured directly from the `source`.
Previously, we had Lock and LockWire impl blocks inter-mixed. This bugs
me a bit, so I've just shuffled things around so that we have Lock, impl
Lock, LockWire and then impl LockWire.
No changes are otherwise made to the code here.
This update follows from the removal of of `source` and `version` from
`distribution.dependency` entries in the lock file when the package name
unambiguously refers to a single distribution.
When there is only one distribution for a particular package name, any
dependencies (the edges in the resolution graph) that reference that
package name are completely unambiguous. Therefore, we can actually omit
their version and source information and instead derive it from the
distribution entry.
We add some tests to check the success and error cases. That is, when
`source` or `version` are omitted and there are more than one
corresponding distribution for the package name (i.e., it's ambiguous),
then lock deserialization should fail.
This commit prepares to make the `source` and `version` fields optional
in a `distribution.dependency` based on whether they have an unambiguous
value. e.g., When there is exactly one distribution with a matching
package name.
This refactor effectively defines "wire" types for most of the lock data
types (repeating the `WheelWire` and `LockWire` pattern) with one key
difference: we don't use serde's `TryFrom` integration. In this
refactor, we could have, and it would have worked. But in a subsequent
commit, we're going to be adding state to the `unwire()` calls that is
impossible to thread through a `TryFrom` implementation. This state will
tell us how to populate the `source` and `version` values on a
`Dependency` when they're missing.
The duplication of types here is unfortunate, but compiler should catch
any deviations. And the wire types are unexported, so they have a
limited blast radius on complexity.
Downstack PR: #4481
## Introduction
We support forking the dependency resolution to support conflicting
registry requirements for different platforms, say on package range is
required for an older python version while a newer is required for newer
python versions, or dependencies that are different per platform. We
need to extend this support to direct URL requirements.
```toml
dependencies = [
"iniconfig @ 62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
"iniconfig @ b3c12c6d70988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'"
]
```
This did not work because `Urls` was built on the assumption that there
is a single allowed URL per package. We collect all allowed URL ahead of
resolution by following direct URL dependencies (including path
dependencies) transitively, i.e. a registry distribution can't require a
URL.
## The same package can have Registry and URL requirements
Consider the following two cases:
requirements.in:
```text
werkzeug==2.0.0
werkzeug @ 960bb4017c4aed12b5ed8b78e0153e/Werkzeug-2.0.0-py3-none-any.whl
```
pyproject.toml:
```toml
dependencies = [
"iniconfig == 1.1.1 ; python_version < '3.12'",
"iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```
In the first case, we want the URL to override the registry dependency,
in the second case we want to fork and have one branch use the registry
and the other the URL. We have to know about this in
`PubGrubRequirement::from_registry_requirement`, but we only fork after
the current method.
Consider the following case too:
a:
```
c==1.0.0
b @ https://b.zip
```
b:
```
c @ https://c_new.zip ; python_version >= '3.12'",
c @ https://c_old.zip ; python_version < '3.12'",
```
When we convert the requirements of `a`, we can't know the url of `c`
yet. The solution is to remove the `Url` from `PubGrubPackage`: The
`Url` is redundant with `PackageName`, there can be only one url per
package name per fork. We now do the following: We track the urls from
requirements in `PubGrubDependency`. After forking, we call
`add_package_version_dependencies` where we apply override URLs, check
if the URL is allowed and check if the url is unique in this fork. When
we request a distribution, we ask the fork urls for the real URL. Since
we prioritize url dependencies over registry dependencies and skip
packages with `Urls` entries in pre-visiting, we know that when fetching
a package, we know if it has a url or not.
## URL conflicts
pyproject.toml (invalid):
```toml
dependencies = [
"iniconfig @ e96292c7f723f1fa332fe4ed6dfbec/iniconfig-1.1.0.tar.gz",
"iniconfig @ b3c12c6d70988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl ; python_version < '3.12'",
"iniconfig @ 62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl ; python_version >= '3.12'",
]
```
On the fork state, we keep `ForkUrls` that check for conflicts after
forking, rejecting the third case because we added two packages of the
same name with different URLs.
We need to flatten out the requirements before transformation into
pubgrub requirements to get the full list of other requirements which
may contain a URL, which was changed in a previous PR: #4430.
## Complex Example
a:
```toml
dependencies = [
# Force a split
"anyio==4.3.0 ; python_version >= '3.12'",
"anyio==4.2.0 ; python_version < '3.12'",
# Include URLs transitively
"b"
]
```
b:
```toml
dependencies = [
# Only one is used in each split.
"b1 ; python_version < '3.12'",
"b2 ; python_version >= '3.12'",
"b3 ; python_version >= '3.12'",
]
```
b1:
```toml
dependencies = [
"iniconfig @ b3c12c6d70988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl",
]
```
b2:
```toml
dependencies = [
"iniconfig @ 62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl",
]
```
b3:
```toml
dependencies = [
"iniconfig @ e96292c7f723f1fa332fe4ed6dfbec/iniconfig-1.1.0.tar.gz",
]
```
In this example, all packages are url requirements (directory
requirements) and the root package is `a`. We first split on `a`, `b`
being in each split. In the first fork, we reach `b1`, the fork URLs are
empty, we insert the iniconfig 1.1.1 URL, and then we skip over `b2` and
`b3` since the mark is disjoint with the fork markers. In the second
fork, we skip over `b1`, visit `b2`, insert the iniconfig 2.0.0 URL into
the again empty fork URLs, then visit `b3` and try to insert the
iniconfig 1.1.0 URL. At this point we find a conflict for the iniconfig
URL and error.
## Closing
The git tests are slow, but they make the best example for different URL
types i could find.
Part of #3927. This PR does not handle `Locals` or pre-releases yet.
In the last PR (#4430), we flatten the requirements. In the next PR
(#4435), we want to pass `Url` around next to `PubGrubPackage` and
`Range<Version>` to keep track of which `Requirement`s added a url
across forking. This PR is a refactoring split out from #4435 that rolls
the dependency conversion into a single iterator and introduces a new
`PubGrubDependency` struct as abstraction over `(PubGrubPackage,
Range<Version>)` (or `(PubGrubPackage, Range<Version>,
VerbatimParsedUrl)` in the next PR), and it removes the now unnecessary
`PubGrubDependencies` abstraction.
This PR refactors the command creation in the test suite to remove the
duplication.
**1)** We add the same set of test stubbing args to almost any uv
invocation in the tests:
```rust
command
.arg("--cache-dir")
.arg(self.cache_dir.path())
.env("VIRTUAL_ENV", self.venv.as_os_str())
.env("UV_NO_WRAP", "1")
.env("HOME", self.home_dir.as_os_str())
.env("UV_TOOLCHAIN_DIR", "")
.env("UV_TEST_PYTHON_PATH", &self.python_path())
.current_dir(self.temp_dir.path());
if cfg!(all(windows, debug_assertions)) {
// TODO(konstin): Reduce stack usage in debug mode enough that the tests pass with the
// default windows stack of 1MB
command.env("UV_STACK_SIZE", (8 * 1024 * 1024).to_string());
}
```
Centralizing these into a `TestContext::add_shared_args` method removes
them from everywhere.
**2)** Prefix all `TextContext` methods of the pip interface with
`pip_`. This is now necessary due to `uv sync` vs. `uv pip sync`.
**3)** Move command creation in the various test files into dedicated
functions or methods to avoid repeating the arguments. Except for error
message tests, there should be at most one `Command::new(get_bin())`
call per test file. `EXCLUDE_NEWER` is exclusively used in
`TestContext`.
---
I'm considering adding a `TestCommand` on top of these changes (in
another PR) that holds a reference to the `TextContext`, has
`add_shared_args` as a method and uses `Fn(Self) -> Self` instead of
`Fn(&mut Self) -> Self` for methods to improved chaining.
Downstack PR: #4515 Upstack PR: #4481
Consider these two cases:
A:
```
werkzeug==2.0.0
werkzeug @ 960bb4017c4aed12b5ed8b78e0153e/Werkzeug-2.0.0-py3-none-any.whl
```
B:
```toml
dependencies = [
"iniconfig == 1.1.1 ; python_version < '3.12'",
"iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```
In the first case, `werkzeug==2.0.0` should be overridden by the url. In
the second case `iniconfig == 1.1.1` is in a different fork and must
remain a registry distribution.
That means the conversion from `Requirement` to `PubGrubPackage` is
dependent on the other requirements of the package. We can either look
into the other packages immediately, or we can move the forking before
the conversion to `PubGrubDependencies` instead of after. Either version
requires a flat list of `Requirement`s to use. This refactoring gives us
this list.
I'll add support for both of the above cases in the forking urls branch
before merging this PR. I also have to move constraints over to this.
While working on https://github.com/astral-sh/uv/pull/4492 I noticed
that `--reinstall-package` was not actually respected by
`update_environment`, it exited early due to satisfied requirements.
Before
```
❯ cargo run -q -- tool install black -v --reinstall-package tomli
...
DEBUG All requirements satisfied: black | click>=8.0.0 | mypy-extensions>=0.4.3 | packaging>=22.0 | pathspec>=0.9.0 | platformdirs>=2 | tomli>=1.1.0 ; python_version < '3.11' | typing-extensions>=4.0.1 ; python_version < '3.11'
```
After
```
❯ cargo run -q -- tool install black -v --reinstall-package tomli
...
Uninstalled 1 package in 0.99ms
Installed 1 package in 4ms
- tomli==2.0.1
+ tomli==2.0.1
```
## Summary
This is an intermediary change in enabling universal resolution for
`requirements.txt` files. To start, we need to be able to preserve
markers in the `requirements.txt` output _and_ propagate those markers,
such that if you have a dependency that's only included with a given
marker, the transitive dependencies respect that marker too.
Closes#1429.
## Summary
If the user puts their configuration in a `pyproject.toml` that _isn't_
a valid workspace root (e.g., it's a Poetry file), we won't discover it,
because we only look in `uv.toml` files in that case. I think this is
somewhat debatable... We could choose to _require_ `uv.toml` there, but
as a user I'd probably expect it to work?
Closes https://github.com/astral-sh/uv/issues/4521.
`junction::create` apparently will happily succeed but not create a link
to files? Since our symlink function does not indicate that it cannot
handle files, this was quite surprising.
Tested over in #4509 which previously failed on an assertion that
`black.exe` existed.
```
error: Failed to install entrypoint
Caused by: Cannot create a junction for [TEMP_DIR]/tools/black/Scripts/black.exe: is not a directory
```
We should file an issue upstream too, I think?
## Summary
We currently accept `--index-url /path/to/index` on the command line,
but confusingly, not in `requirements.txt`. This PR just brings the two
in sync.
## Test Plan
New snapshot tests.
## Summary
pip allows these with the following logic:
```python
if os.path.exists(location): # Is a local path.
url = path_to_url(location)
path = location
elif location.startswith("file:"): # A file: URL.
url = location
path = url_to_path(location)
elif is_url(location):
url = location
```
Closes https://github.com/astral-sh/uv/issues/4510.
## Test Plan
`cargo run pip install --index-url ../packse/index/simple-html/
example-a-961b4c22 --reinstall --no-cache --no-deps`
I was getting this test failure locally on my Archlinux system:
```
-old snapshot
+new results
0 0 │ success: true
1 1 │ exit_code: 0
2 2 │ ----- stdout -----
3 │-[PYTHON-3.12]
3 │+/usr/bin/python3
4 4 │
5 5 │ ----- stderr -----
```
Where I have `/usr/bin/python3` and `/usr/bin/python3.12`.
Thanks @zanieb for the help with figuring out the fix here!
## Summary
In #4085, support was implemented for the `--prefix` option. When using
this option, however, a lock is either acquired on the virtualenv or
globally, preventing multiple installs to different `--prefix`s from the
same interpreter.
In this change, acquire the lock on just the prefix in question.
## Test Plan
Ran a `uv pip install` with `--prefix` and `RUST_LOG=trace` and observed
that the lock was acquired in the prefix.
## Summary
It turns out that the Git fetch implementation is initializing its own
client, which can be really expensive on macOS (due to loading native
certificates) _and_ bypasses any of our middleware. This PR modifies the
Git implementation to accept a shared client.
## Summary
We unconditionally update the submodules in our Git code, but AFAICT it
shouldn't be necessary if we already have a complete, up-to-date fetch
available.
## Summary
This does require cloning the settings, but I think it's fine. A better
solution would be to have owned and unowned settings structs, so that we
could convert `ResolverInstallerSettingsRef` to `InstallerSettingsRef`
without cloning, but that requires maintaining owned and unowned
variants.
Closes https://github.com/astral-sh/uv/issues/4455.
## Summary
I might be mistaken, but I think we need to read the header from the
response, not the request. The request would only contain headers that
we set.
I verified (with extra logging) that the request header is `None` while
PyPI returns a valid length in the response header.
## Summary
Ultimately decided to view this as part of `LockWire` normalization:
removing references to extras that don't exist. I think it would be nice
if the resolver avoided omitting these, but I don't know if it's fully
possible.
Closes https://github.com/astral-sh/uv/issues/4405.
I have to add yet another indentation level to the prerelease-available
check in `PubGrubReportFormatter::hints` for #4435, so i've broken the
code into methods and decreased indentation in this split out
refactoring-only change.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Resolves https://github.com/astral-sh/uv/issues/4439 partially.
Implements for `uv pip tree`:
- `--no-dedupe` flag, similar to `cargo tree --no-dedupe` .
- denote dependency cycles with `(#)` and add a footnote if there's a
cycle (using `(*)` would require keeping track of the cycle state, so
opted to do this instead).
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
The existing tests pass + added a couple of tests to validate
`--no-dedupe` behavior.
<!-- How was it tested? -->
## Summary
This will make `uv lock` read `override-dependencies` from the
`[tool.uv]` section of `pyproject.toml`.
Resolves#4108
This [other](https://github.com/astral-sh/uv/pull/4446) implementation
touches more code but seems more consistent.
## Test Plan
Unit test
Releasing 0.2.15 with a few additions over 0.2.14. Motivated by the
incorrect tagging of 0.2.14 (#4474).
Generated the changelog with a small patch to Rooster allowing me to
force the previous commit to be correct.
```diff
diff --git a/src/rooster/_cli.py b/src/rooster/_cli.py
index 2a4f61b..4ec1299 100644
--- a/src/rooster/_cli.py
+++ b/src/rooster/_cli.py
@@ -38,6 +38,7 @@ def release(
without_sections: list[str] = typer.Option(
[], help="Sections to exclude from the changelog"
),
+ previous_commit: str = None,
):
"""
Create a new release.
@@ -58,7 +59,11 @@ def release(
typer.echo("It looks like there are no version tags for this project.")
# Get the commits since the last release
- changes = list(get_commits_between(config, repo, last_version))
+ changes = list(
+ get_commits_between(
+ config, repo, last_version, force_first_commit=previous_commit
+ )
+ )
since = "since last release" if last_version else "in the project"
typer.echo(f"Found {len(changes)} commits {since}.")
diff --git a/src/rooster/_git.py b/src/rooster/_git.py
index 597bb88..66bc54e 100644
--- a/src/rooster/_git.py
+++ b/src/rooster/_git.py
@@ -29,12 +29,13 @@ def get_commits_between(
target: Path,
first_version: Version | None = None,
second_version: Version | None = None,
+ force_first_commit: str | None = None,
) -> Generator[git.Commit, None, None]:
"""
Yield all commits between two tags
"""
repo = git.repository.Repository(target.absolute())
- first_commit = (
+ first_commit = force_first_commit or (
repo.lookup_reference(
TAG_PREFIX + config.version_tag_prefix + str(first_version)
)
```
## Summary
The `--index-strategy` is linked to the index locations, which we
propagate to source distribution builds; so it makes sense to pass the
`--index-strategy` too.
While I was here, I made `exclude_newer` a required argument so that we
don't forget to set it via the `with_options` builder.
Closes https://github.com/astral-sh/uv/issues/4465.
## Summary
This PR moves all the CLI code into its own crate, separate from the
`uv` crate. The `uv` crate is iterated on frequently, and the CLI code
comprises a significant portion of it but rarely changes. Removing the
CLI code reduces the `uv` crate size from 1.4MiB to 1.0MiB.
When executables were not named `python3` e.g. `python3.11` we would
construct a Python path that would only work for _some_ requests in
tests since we don't search for those names unless a specific version is
requested. To solve, we construct a test context with constant Python
executable names. For example, if a test context was created with `3.11`
and `3.12` we could end up with the search path
`/usr/local/python-3.11/bin:/usr/local/python-3.12/bin` where the
executables are named `python3.11` and `python3` respectively. A test
invocation of uv requesting any Python toolchain version would then
locate the `3.12` executable since the `3.11` executable doesn't have
the generic name, but we want `3.11` to come first.
On Windows, we just leave things as-is because executables are always
called `python.exe`.
Closes https://github.com/astral-sh/uv/issues/4376
## Summary
Closes https://github.com/astral-sh/uv/issues/2956
This changes the bootstrap launcher script to use `pythonw.exe` instead
of `python.exe` on `gui_scripts` via a helper fn both in the shebang and
the python exe path encoded before `UVUV` magic, that way
uv-trampoline's `find_python_exe` can use the right pythonw executable.
## Test Plan
New unit tests for the helper was added.
Tested on example from #2956 on Windows to make sure it works as
expected.
## Questions
I noticed the docs in `fn windows_script_launcher` says ```The launcher
will look for `python[w].exe` adjacent to it in the same directory to
start the embedded script.``` but I didn't find such functionality when
I looked in uv-trampoline.
I only saw `clear_app_starting_state` getting called when `is_gui` is
set.
Was the intention to do this in uv-trampoline at some point instead?
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
resolves https://github.com/astral-sh/uv/issues/3272
added it as a new subcommand rather than a flag on an existing
command since that seems more consistent with `cargo tree` + cleaner
code organization, but can make changes if it's preferred the other way.
Exposes the option added in #4416. Adds `--toolchain-preference` and
`tool.uv.toolchain-preference` to configure if system or managed
toolchains are preferred. Users can opt-out of managed toolchains or
system toolchains entirely as well.
Restores the `PythonEnvironment::find` API which was removed a while
back in favor of `Toolchain::find`. As mentioned in #4416, I'm
attempting to separate the case where you want an active environment
from the case where you want an installed toolchain in order to create
environments.
I wanted to drop `EnvironmentPreference` from `Toolchain::find` and just
have us consistently consider (or not consider) virtual environments
when discovering toolchains for creating environments. Unfortunately
this caused a few things to break so I reverted that change and will
explore it separately. Because I was exploring that change, there are
some minor changes to the `Toolchain` API here.
Adds support for the toolchain discovery preferences outlined in
https://github.com/astral-sh/uv/issues/4198 but we don't expose this to
users yet, I'll do that next to make it easier to review.
I've made some refactors in the toolchain discovery implementation to
enable this behavior and move us towards clearer abstractions. There's
still remaining work here, but I'd prefer tackle things in follow-ups
instead of expanding this pull request. I plan on opening a couple
before merging this.
I'd like to shift the public toolchain API to focus on discovering
either an **environment** or a **toolchain**. The first would be used by
commands that operate on an environment, while the latter would be used
by commands that just need an interpreter to create environments. I
haven't changed this here, but some of the refactors are in preparation
for supporting this idea.
In brief:
- We now allow different ordering of installed toolchain discovery based
on a `ToolchainPreference` type. This is the type we will expose to
users.
- `SystemPython` was changed into an `EnvironmentPreference` which is
used to determine if we should prefer virtual or system Python
environments.
- We drop the whole `ToolchainSources` selection concept, it was
confusing and the error messages from it were awkward. Most of the
functionality is now captured by the preference enums, but you can't do
things like "only find a toolchain from the parent interpreter" as
easily anymore.
Previously, distributions created through `Source::Workspace` would have
the absolute path as lock path. This didn't cause any problems, since in
`Urls` we would later overwrite those urls with the correct one created
from being workspace members by path.
Changing the order surfaced this. This change emits the correct lock
path. I've manually checked the difference with `dbg!`, this is not
observable on main, but on the diverging urls branch it fixes lockfile
creation.
When a fork is created from a list of dependencies, we were previously
adding all other sibling dependencies to every fork created. But this
isn't actually quite right, since the fork created is always created by
some marker expression. And while it is definitively disjoint from any
directly conflicting dependency specification, it is also possibly
disjoint with other dependencies. For example, as reported in #4414:
```toml
dependencies = [
"anyio==4.4.0 ; python_version >= '3.12'",
"anyio==4.3.0 ; python_version < '3.12'",
"b1 ; python_version >= '3.12'",
"b2 ; python_version < '3.12'",
]
```
The first two `anyio` requirements are conflicting with non-overlapping
marker expressions, and so a fork is created. Prior to this commit,
*both* `b1` and `b2` would be added to each fork. But of course, `b2` is
impossible in the `anyio==4.4.0` fork because of disjoint marker
expressions.
So in this commit, we specifically filter out any sibling dependencies
that could find their way into a fork that have disjoint markers with
that fork. We are careful to do this both when a new fork is created
from an existing set of dependencies, and when adding new dependencies
to a fork.
Fixes#4414
## Summary
After this change, `uv add` will try to use `tool.uv.sources` for all
source requirements. If a source cannot be resolved, i.e. an ambiguous
Git reference is provided, it will error. Git references can be
specified with the `--tag`, `--branch`, or `--rev` arguments. Editables
are also supported with `--editable`.
Users can opt-out of `tool.uv.sources` support with the `--raw` flag,
which will force uv to use `project.dependencies`.
Part of https://github.com/astral-sh/uv/issues/3959.
To support diverging urls, we have to check urls when adding
dependencies (after forking). To prepare for this, i've moved adding
dependencies for the current version to
`SolveState::add_package_version_dependencies` and removed the
duplication when checking for self-dependencies.
This changed is joined with a change in pubgrub
(https://github.com/astral-sh/pubgrub/pull/27) that simplifies the same
code path.
## Summary
Implements `uv add foo --workspace`, which adds `foo` as a workspace
dependency with the corresponding `tool.uv.sources` entry.
Part of https://github.com/astral-sh/uv/issues/3959.
Specifically, these are emitted when requirements fail to satisfy
`Requires-Python` or the markers associated with the current fork in the
resolver.
Closes#4373
Closes#4380
This is the same logic as `should_stop_discovery` but I changed the log
level and duplicated it because I don't really want that method to be
public. Maybe it should be though?
As in #4360, updates the uv project CLI to respect `.python-version`
files as default Python version requests. Additionally, updates project
interpreter discovery to fetch managed toolchains as in `uv venv
--preview`.
It was becoming problematic that the virtual environment test context
diverged from the other one i.e. we had to implement filtering twice.
This combines the contexts and tweaks the `TestContext` API and
filtering mechanisms for Python versions. Combined with my previous
changes to the test context at #4364 and
https://github.com/astral-sh/uv/pull/4368 this finally unblocks the
snapshots for test cases in #4360 and
https://github.com/astral-sh/uv/pull/4362.
## Summary
Make the git commit/date part of the version string matched in
cache_prune optional, so that the test also works correctly when uv is
built from an unpacked release tarball rather than a git repository.
Fixes#4374
## Test Plan
Ran `cargo test` inside the git repository and inside unpacked archive
for `0.2.12` release with the patch applied on top.
## Summary
Adds a `--no-clean` flag to `uv sync` that keeps extraneous
installations. This is the default in `uv run` and `uv add`, but not in
`uv sync` or `uv remove`. This means you need to run an explicit `uv
sync/remove` to clean the virtual environment.
Extends new filters for interpreter paths to apply to tests with
multiple Python versions. Adds patch version filtering for them as well,
which is needed for #4360 tests.
Extends #4364 automatically adding filters to the test context for
additional test virtual environments.
It turns out that the `pip sync` tests were really on the loose with
their virtual environment creation and it was difficult to use the new
helpers because they require mutability and the tests had immutable
borrows to extend the filters 😭. I refactored the tests to just reset
the environment.
A bare `uv toolchain install` invocation now reads default requests from
Python version files in the working directory. In order, a bare
invocation means:
- requests from `.python-versions`
- a single request from`.python-version`
- any installed managed toolchain
- the latest managed toolchain download
This replaces all the functionality of `cargo dev fetch-python`, which
we drop in #4337
## Summary
Closes https://github.com/astral-sh/uv/issues/4162
Changes keyring subprocess to allow display of stderr.
This aligns with pip's behavior since pip 23.1.
## Test Plan
* Tested using gnome-keyring-backend on a self-hosted private registry
as well as the keyring script described in #4162 to confirm both
existing functionality and the new stderr display.
* Existing tests using `scripts/packages/keyring_test_plugin` are now
showing its stderr output as well.
Allows installation of multiple toolchains in a single invocation
because I don't want to be limited to one! Most of the implementation
for concurrent downloads ported from `cargo dev fetch-python`.
Adds support for toolchain keys e.g. `cpython-3.11.2-macos` allowing you
to download toolchains for specific architectures and operating systems
using the format we use to uniquely identify a toolchain.
Closes#4240
e.g.
```
❯ cargo run -q -- pip install anyio --python "/Users/zb/Library/Application Support/uv/toolchains/cpython-3.12.0-macos-aarch64-none/install/bin/python3"
error: The interpreter at /Users/zb/Library/Application Support/uv/toolchains/cpython-3.12.0-macos-aarch64-none/install is externally managed, and indicates the following:
This toolchain is managed by uv and should not be modified.
Consider creating a virtual environment with `uv venv`.
```
I found this while testing the tracking of marker expressions across
resolver forks. Namely, given
sys_platform == 'darwin' and implementation_name == 'pypy'
And:
sys_platform == 'bar' or implementation_name == 'foo'
These should be disjoint, but the disjointness checker was reporting
them as overlapping. I fixed this by giving handling of disjunctions
higher precedence than conjunctions, although I am not 100% confident
that this is correct for all cases.
This commit adds marker expressions to our `Fork` type, which are in
turn passed down into `PubGrubDependencies::from_requirements` to filter
our any dependencies with markers that are disjoint from the fork's
marker expression.
This is necessary to avoid visiting packages in the dependency graph
that can never actually be installed. This is because when a fork is
created in the resolver, it always happens when there are two sibling
dependency specifications on a package with the same name, but with
non-overlapping marker expressions. Each fork corresponds to each
such conflicting dependency specification, and each fork assumes the
the corresponding marker expression as a pre-condition for any future
dependencies considered by it. That is, since the fork represents an
installation path that can only be taken when the corresponding
dependency specification (and its marker expression) is actually used,
it also therefore follows that the marker expression is true. Therefore,
any dependency visited in that fork with a marker expression that cannot
possibly be true when the markers of the fork are true can and ought to
be completely ignored.
There are some key invariants that I had to re-learn by reading the
code. This hopefully makes those invariants easier to discover by future
me (and others).
Otherwise, when testing `uv venv --preview` the default toolchain
directory will leak into the test.
I believe I've made a similar change for the standard `TestContext` in
another commit somewhere in my stack, if not I'll add it after.
Adds a command to find a toolchain on the system. Right now, it displays
the path to the first matching toolchain. We'll probably have more rich
output in the future (after implementing `toolchain show`).
The eventual plan (separate from here) is to port all of the toolchain
discovery tests to use this command. I'll add a few tests for this
command here anyway.
## Summary
This would be a lightweight solution to
https://github.com/astral-sh/uv/issues/4307 that doesn't fully engage
with all the possibilities in the design space (but would unblock
cross-platform for now).
## Summary
The fixtures here are pretty large, but it lets us test what we actually
care about (the resolved settings) rather than inferring the resolved
settings from behavior, which I think is a big improvement.
I also broke the tests down into more granular cases.
## Summary
Because the workspace member itself is part of the resolution, adding
the workspace name for the project leads to confusing errors, like:
```
❯ cargo run lock --preview
Compiling uv v0.2.11 (/Users/crmarsh/workspace/puffin/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.79s
Running `/Users/crmarsh/workspace/puffin/target/debug/uv lock --preview`
× No solution found when resolving dependencies:
╰─▶ Because only albatross==0.1.0 is available and albatross==0.1.0 depends on anyio<=3, we can conclude that all versions of albatross depend on anyio<=3.
And because bird-feeder==1.0.0 depends on anyio>=4.3.0,<5 and only bird-feeder==1.0.0 is available, we can conclude that all versions of albatross and all versions of bird-feeder are incompatible.
And because albatross depends on albatross and bird-feeder, we can conclude that the requirements are unsatisfiable.
```
(Notice "albatross depends on albatross".)
## Summary
In a workspace, we now read configuration from the workspace root.
Previously, we read configuration from the first `pyproject.toml` or
`uv.toml` file in path -- but in a workspace, that would often be the
_project_ rather than the workspace configuration.
We need to read configuration from the workspace root, rather than its
members, because we lock the workspace globally, so all configuration
applies to the workspace globally.
As part of this change, the `uv-workspace` crate has been renamed to
`uv-settings` and its purpose has been narrowed significantly (it no
longer discovers a workspace; instead, it just reads the settings from a
directory).
If a user has a `uv.toml` in their directory or in a parent directory
but is _not_ in a workspace, we will still respect that use-case as
before.
Closes#4249.
## Summary
This PR introduces top-level configuration for uv, such that you can do:
```toml
[tool.uv]
index-url = "https://test.pypi.org/simple"
```
And `uv pip compile`, `uv run`, `uv tool run`, etc., will all respect
that configuration.
The settings that were escalated to the top-level remain on
`tool.uv.pip` too, but they're only respected in `uv pip` commands. If
they're specified in both places, then the `pip` settings win out.
While making this change, I also wired up some of the global options,
like `connectivity` and `native_tls`, through to all the relevant
places.
Closes#4250.
Previously, we took the first executable on the `PATH` but if it was not
a usable interpreter we'd fail. Now, we'll continue searching in the
path until we find an interpreter as we do with the standard executable
names.
Previously, `b` in the test case would have been incorrectly locked to
the path of `a`. I've moved `relative_to` into uv-fs since it's now used
in two different places.
Previously failing lockfile when `a/pyproject.toml` and
`a/b/pyproject.toml` exist (not in a workspace) and `a` was depending on
`b`:
```toml
version = 1
requires-python = ">=3.11, <3.13"
[[distribution]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
sdist = { path = "/home/konsti/projects/uv/a" }
[[distribution]]
name = "black"
version = "0.1.0"
source = "editable+."
sdist = { path = "." }
[[distribution.dependencies]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
```
Includes system interpreters in `uv toolchain list`.
This includes a refactor of `find_toolchain` to support iterating over
all toolchains
that match a request rather than ending earlier.
## Summary
This i still a draft, but it gets some of the work started on issue
#4064 , which requests --no-binary functionality similar to `uv pip
install` to exist on `uv pip compile`.
So far this has moved the command line shape from install and cloned it
over to compile. The actual functionality of respecting --no-binary
<package> and generating the resulting line in requirements.txt I could
use a hand with.
My understanding is we want to create a requirements.in file such as:
```
yt
```
Then when running `cargo run -- pip compile --no-binary yt
requirements.in`
we want the file to have this line in it:
```
yt==4.3.1 --no-binary yt
```
## Test Plan
Existing unit tests continue to pass.
No new unit tests have been created yet.
The new command line options do show up when testing with `cargo run --
pip compile -h`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Closes https://github.com/astral-sh/uv/issues/4296.
## Test Plan
Ran `cargo run lock --verbose` from
`scripts/workspaces/albatross-virtual-workspace`:
```
DEBUG uv 0.2.11 (ef3bc1612 2024-06-12)
warning: `uv lock` is experimental and may change without warning.
DEBUG Found workspace root: `/Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace`
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/albatross
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/seeds
DEBUG Searching for Python >=3.12 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/crmarsh/Library/Application Support/uv/toolchains`
DEBUG Found managed toolchain `cpython-3.12.3-macos-aarch64-none`
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3` (managed toolchains)
Using Python 3.12.3 interpreter at: /Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3
```
Before 0.2.10 we would parse `--python=python` as an executable name.
After https://github.com/astral-sh/uv/pull/4214, we started treating
this as a Python version range request (with an empty version range).
This is not entirely unreasonable, but it was an unexpected regression
and I don't think `VersionRequest` should support empty ranges in its
`from_str` implementation without more consideration.
Updates `--no-binary <package>` to take precedence over `--only-binary
:all:` and `--only-binary <package>` to take precedence over
`--no-binary :all:`.
I'm not entirely sure about this behavior, e.g. maybe I provided
`--only-binary :all:` later on the command line and really want it to
override those earlier arguments of `--no-binary <package>` for safety.
Right now we just fail to solve though since we can't satisfy the
overlapping requests.
Closes https://github.com/astral-sh/uv/issues/4063
## Summary
This is what I consider to be the "real" fix for #8072. We now treat
directory and path URLs as separate `ParsedUrl` types and
`RequirementSource` types. This removes a lot of `.is_dir()` forking
within the `ParsedUrl::Path` arms and makes some states impossible
(e.g., you can't have a `.whl` path that is editable). It _also_ fixes
the `direct_url.json` for direct URLs that refer to files. Previously,
we wrote out to these as if they were installed as directories, which is
just wrong.
We weren't using the common interface in `uv lock` because it didn't
support finding an interpreter without touching the virtual environment.
Here I refactor the project interface to support what we need and update
`uv lock` to use the shared implementation.
In the time before universal resolving, we would always pass a
`MarkerEnvironment`, and this environment would capture any relevant
`Requires-Python` specifier (including if `-p/--python` was provided on
the CLI).
But in universal resolution, we very specifically do not use a
`MarkerEnvironment` because we want to produce a resolution that is
compatible across potentially multiple environments. This in turn meant
that we lost `Requires-Python` filtering.
This PR adds it back. We do this by converting our `PythonRequirement`
into a `MarkerTree` that encodes the version specifiers in a
`Requires-Python` specifier. We then ask whether that `MarkerTree` is
disjoint with a dependency specification's `MarkerTree`. If it is, then
we know it's impossible for that dependency specification to every be
true, and we can completely ignore it.
## Summary
Right now, we're _always_ reinstalling local wheel archives, even if the
timestamp didn't change.
I want to fix the TODO properly but I will do so in a separate PR.
## Summary
`normalize` now takes an owned value and returns `Option<MarkerTree>`,
such that if any sub-expression evaluates to `true`, we can normalize
out the entire marker.
Closes https://github.com/astral-sh/uv/issues/4267.
Closes https://github.com/astral-sh/uv/issues/3857
Instead of using custom `Arch`, `Os`, and `Libc` types I just use
`target-lexicon`'s which enumerate way more variants and implement
display and parsing. We use a wrapper type to represent a couple special
cases to support the "x86" alias for "i686" and "macos" for "darwin".
Alternatively we could try to use our `platform-tags` types but those
capture more information (like operating system versions) that we don't
have for downloads.
As discussed in https://github.com/astral-sh/uv/pull/4160, this is not
sufficient for proper libc detection but that work is larger and will be
handled separately.
## Summary
Fix the docsting where `remove` was used instead of `add` in the context
of `uv add` command.
## Test Plan
```
cargo run -- add --help
```
```
Add one or more packages to the project requirements
Usage: uv add [OPTIONS] <REQUIREMENTS>...
Arguments:
<REQUIREMENTS>...
The packages to add, as PEP 508 requirements (e.g., `flask==2.2.3`)
```
## Summary
I think this is a useful piece of connective tissue that will let us
avoid back-and-forths when folks include traces.
## Test Plan
```
❯ cargo run pip list --verbose
DEBUG uv 0.2.11 (44041bccd 2024-06-11)
DEBUG Searching for Python interpreter in virtual environments
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/workspace/puffin/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
```
## Summary
If we're ORing an OR, we should just append rather than nesting in
another OR.
In my branch, this let us simplify:
```
python_version < '3.10' or python_version > '3.12' or (python_version < '3.8' or python_version > '3.12')
```
To:
```
python_version < '3.10' or python_version > '3.12
```
## Summary
The build tags in this case are like, e.g., `202206090410`. That's
larger than a `u32`, so we're rejecting the wheel. In theory build tags
could be even larger, but we already use `u64` for version segment so I
think it's fine to keep that constraint here.
I'm going to look into surfacing these errors separately.
Closes https://github.com/astral-sh/uv/issues/4252.
## Test Plan
`cargo run pip install monailabel`
e.g.
```
❯ uv toolchain install
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
A toolchain is already installed. Use `uv toolchain install <request>` to install a specific toolchain
```
instead of
```
❯ uv toolchain install
Using latest Python version
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
Already installed at /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.19-macos-aarch64-none
```
## Summary
Something that looks like it was forgotten to replace in #4164.
## Test Plan
Run `cargo run toolchain install` should display the warning: ``warning:
`uv toolchain install` is experimental and may change without warning.``
By splitting `path` into a lockable, relative (or absolute) and an
absolute installable path and by splitting between urls and paths by
dist type, we can store relative paths in the lockfile.
## Summary
We may choose to persist these eventually, but for now, it's useful to
have them colocated with the cache, and in their own dedicated bucket
(so, at the very least, we can keep track of the use-cases).
Closes https://github.com/astral-sh/uv/issues/4219.
A merge kerfuffle from #4222 and #4218
Now we fail because we genuinely can't find any interpreters since tests
contexts are isolated by default. I'll improve the error message and
maybe add another test case once `main` is fixed.
By setting the test search path to an empty path, we avoid accidentally
pulling interpreters from the system during a test case.
Cherry-picked from https://github.com/astral-sh/uv/pull/4214
Cherry-picked from https://github.com/astral-sh/uv/pull/4214
The first commit gets us some context on an IO error during queries:
Previously:
```
failed to canonicalize path `[VENV]/bin/python3`
Caused by: No such file or directory (os error 2)
```
Now:
```
Failed to query Python interpreter
Caused by: failed to canonicalize path `[VENV]/bin/python3`
Caused by: No such file or directory (os error 2)
```
but really we shouldn't attempt to query a missing interpreter during
discovery anyway, so we improve handling of that too.
## Summary
Should be no behavior changes, but one piece of technical debt I noticed
left over in the URL resolver. We already have structured paths, so we
shouldn't need to compare verbatim URLs.
## Summary
If the user requests a package as both editable and non-editable, the
editable now "wins".
Previously, `pip install -e . .` would install as editable. However,
`pip install -e . -r requirements.txt` would _not_ if `requirements.txt`
contained `.`, because we ignored `editable` when deduplicating and the
order of iteration was just dependent on internals.
Closes https://github.com/astral-sh/uv/issues/4053.
Adds a command (following #4163) to download and install specific
toolchains. While we fetch toolchains on demand, this is useful for,
e.g., pre-downloading a toolchain in a Docker image build.
~I kind of think we should call this `install` instead of `fetch`~ I
changed the name from `fetch` to `install`.
## Summary
Adds handling for a few cases to improve interoperability with Poetry:
- If the `project` schema is invalid, we now raise a hard error, rather
than treating the metadata as dynamic and then falling back to the build
backend. This could cause problems, I'm not sure. It's stricter than
before.
- If the project contains `tool.poetry` but omits
`project.dependencies`, we now treat it as dynamic. We could go even
further and treat _any_ Poetry project as dynamic, but then we'd be
ignoring user-declared dependencies, which is also confusing.
Closes https://github.com/astral-sh/uv/issues/4142.
Extends https://github.com/astral-sh/uv/pull/4121
Part of #2607
Adds support for managed toolchain fetching to `uv venv`, e.g.
```
❯ cargo run -q -- venv --python 3.9.18 --preview -v
DEBUG Searching for Python 3.9.18 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/zb/Library/Application Support/uv/toolchains`
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Found CPython 3.9.6 at `/usr/bin/python3` (search path)
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Requested Python not found, checking for available download...
DEBUG Using registry request timeout of 30s
INFO Fetching requested toolchain...
DEBUG Downloading https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst to temporary location /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp
DEBUG Extracting cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst
DEBUG Moving /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp/python to /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none
Using Python 3.9.18 interpreter at: /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
INFO Removing existing directory
Activate with: source .venv/bin/activate
```
The preview flag is required. The fetch is performed if we can't find an
interpreter that satisfies the request. Once fetched, the toolchain will
be available for later invocations that include the `--preview` flag.
There will be follow-ups to improve toolchain management in general,
there is still outstanding work from the initial implementation.
## Summary
Similar to how we abstracted the dependencies into
`ResolutionDependencyNames`, I think it makes sense to abstract the base
packages into a `ResolutionPackage`. This also avoids leaking details
about the various `PubGrubPackage` enum variants to `ResolutionGraph`.
Remove the panic when there is an invalid wheel source, instead surface
the error. This error can only occur when manually editing the lock
file, but since it's an external file, we should error and not panic.
This change is helpful since the method needs to be able to error for
relative path support.
## Summary
If a package lacks a source distribution, and we can't find a compatible
wheel for the current platform, we need to just _assume_ that the
package will have a valid wheel on all platforms on which it's
requested; if not, we raise an error at install time.
It's possible that we can be smarter about this over time. For example,
if the package was requested _only_ for macOS, we could verify that
there's at least one macOS-compatible wheel. See the linked issue for
more details.
Closes https://github.com/astral-sh/uv/issues/4139.
## Summary
If we want more granular control over how these errors are handled, then
we need to move them out of the TOML deserialization.
No actual behavior changes here.
Part of https://github.com/astral-sh/uv/issues/4142.
## Summary
See the long comment inline. I think this is debatable but probably
right for now. The other options have their own problems, but there are
a few alternate ideas in the comment.
Closes https://github.com/astral-sh/uv/issues/4132.
The basic idea here is to make it so forking can only ever result in a
resolution that, for a particular marker environment, will only install
at most one version of a package. We can guarantee this by ensuring we
only fork on conflicting dependency specifications only when their
corresponding markers are completely disjoint. If they aren't, then
resolution _must_ find a single version of the package in the
intersection of the two dependency specifications.
A test for this case has been added to packse here:
https://github.com/astral-sh/packse/pull/182. Previously, that test
would result in a resolution with two different unconditional versions
of the same package. With this change, resolution fails (as it should).
A commit-by-commit review should be helpful here, since the first commit
is a refactor to make the second commit a bit more digestible.
## Summary
I think we should be able to model PubGrub such that this isn't
necessary (at least for the case described in the issue), but for now,
let's just avoid attempting to build very old distributions in
prefetching.
Closes https://github.com/astral-sh/uv/issues/4136.
Drops `find_toolchain`, `find_best_toolchain`, etc. in favor of
`Toolchain::find_...`
We can change this in the future, but there should only be one "right"
way to do it not two redundant ways in the public interface.
## Summary
Simplify and normalize marker expressions in the lockfile. Right now
this does a simple analysis by only looking at related operators at the
same level of precedence. I think anything more complex would be out of
scope.
Resolves https://github.com/astral-sh/uv/issues/4002.
## Summary
This PR adds a lowering similar to that seen in
https://github.com/astral-sh/uv/pull/3100, but this time, for markers.
Like `PubGrubPackageInner::Extra`, we now have
`PubGrubPackageInner::Marker`. The dependencies of the `Marker` are
`PubGrubPackageInner::Package` with and without the marker.
As an example of why this is useful: assume we have `urllib3>=1.22.0` as
a direct dependency. Later, we see `urllib3 ; python_version > '3.7'` as
a transitive dependency. As-is, we might (for some reason) pick a very
old version of `urllib3` to satisfy `urllib3 ; python_version > '3.7'`,
then attempt to fetch its dependencies, which could even involve
building a very old version of `urllib3 ; python_version > '3.7'`. Once
we fetch the dependencies, we would see that `urllib3` at the same
version is _also_ a dependency (because we tack it on). In the new
scheme though, as soon as we "choose" the very old version of `urllib3 ;
python_version > '3.7'`, we'd then see that `urllib3` (the base package)
is also a dependency; so we see a conflict before we even fetch the
dependencies of the old variant.
With this, I can successfully resolve the case in #4099.
Closes https://github.com/astral-sh/uv/issues/4099.
Extends #4120
Part of #2607
There should be no behavior changes here. Restructures the discovery API
to be focused on a toolchain first perspective in preparation for
exposing a `find_or_fetch` method for toolchains in
https://github.com/astral-sh/uv/pull/4138.
Partially addresses https://github.com/astral-sh/uv/issues/4056
We were incorrectly omitting the port from requests to `keyring` when
falling back to a realm/host query, e.g. `localhost` was used instead of
`localhost:1234`. We still won't include "standard" ports like `80` for
an HTTP request.
Follow-up to #4016.
This exposes `Range` and `PubGrubSpecifier` from outside the resolver to
use pubgrub's union creating a dependency edge we don't really want.
When creating a lockfile, lock the combined dependencies for all
packages in a workspace. This make the lockfile independent of where you
are in the workspace.
Fixes#3983
## Summary
This PR modifies our `Requires-Python` handling to treat
`Requires-Python` as a lower bound. There's extensive discussion around
this in https://github.com/astral-sh/uv/issues/4022 and the references
linked therein. I think it's an experiment worth trying. Even in my own
small projects, I'm running into issues whereby I'm being "forced" to
add a `<4` upper bound to my `Requires-Python` due to these caps.
Separately, we should explore adding a mechanism that's distinct from
`Requires-Python` to enable users to declare a supported range for
locking.
Closes https://github.com/astral-sh/uv/issues/4022.
## Summary
The condition enforced here isn't quite right. The same dependency can
appear multiple times, as long as the extra is different.
Closes https://github.com/astral-sh/uv/issues/4101.
## Summary
The "only include if relevant for the extra" filtering should _not_ be
applied to constraints. Otherwise, we'd only constrain when the extra
was included in the constraints file itself, which is incorrect.
Closes https://github.com/astral-sh/uv/issues/4091.
Charlie Marsh
Zanie Blue
eth3lbert
Andrew Gallant
Ben Beasley
konsti
Maksim Bondarenkov
bluss
Pavel Dikov
Krishnan Chandra
Jo
Ahmed Ilyas
Ibraheem Ahmed
Trevor Manz
samypr100
Zack Elia
Yorick
Di-Is
Chan Kang
Mathieu Kniewallner
Danny
Tim Felgentreff
Silvano Cerza
Ivan Shcheklein
skshetry
messense
Alexander Gherm
renovate[bot]
Caíque Porfirio
Billy Doyle
pppp56
David Poznik
Gilles Peiffer
Eric Mark Martin
Sandu
Michał Górny
Brian Mego
Nicolas Delaby
Nyakku Shigure