In https://github.com/astral-sh/uv/issues/8155#issuecomment-2508969900,
resolution lowest was complaining about missing lower bounds for a
pacakge, even though the package had a URL, too:
```
uv pip install dist/pymatgen-2024.10.3.tar.gz pymatgen[ci,optional] --resolution=lowest
```
The error was raised from `pymatgen[ci,optional]`, because we were
looking at it before looking at the "URL"
`dist/pymatgen-2024.10.3.tar.gz`.
I've also added constraints and overrides to the bounds lookup, since
they are missing from the dependency graph.
Fixes#8155 (again)
## Summary
Closes#9643.
I modified the `commit` fn so this applies to `uv compile --output-file`
too. But I can move it to the export module if we want to restrict this
to `uv export` only.
## Test Plan
`cargo test`
This is like #9556, but at the level of all other builds, including the
resolver and installer. Going through PEP 517 to build a package is
slow, so when building a package with the uv build backend, we can call
into the uv build backend directly instead: No temporary virtual env, no
temp venv sync, no python subprocess calls, no uv subprocess calls.
This fast path is gated through preview. Since the uv wheel is not
available at test time, I've manually confirmed the feature by comparing
`uv venv && cargo run pip install . -v --preview --reinstall .` and `uv
venv && cargo run pip install . -v --reinstall .`. When hacking the
preview so that the python uv build backend works without the setting
the direct build also (wheel built with `maturin build --profile
profiling`), we can see the perfomance difference:
```
$ hyperfine --prepare "uv venv" --warmup 3 \
"UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview" \
"target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/"
Benchmark 1: UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview
Time (mean ± σ): 33.1 ms ± 2.5 ms [User: 25.7 ms, System: 13.0 ms]
Range (min … max): 29.8 ms … 47.3 ms 73 runs
Benchmark 2: target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/
Time (mean ± σ): 115.1 ms ± 4.3 ms [User: 54.0 ms, System: 27.0 ms]
Range (min … max): 109.2 ms … 123.8 ms 25 runs
Summary
UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview ran
3.48 ± 0.29 times faster than target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/
```
Do we need a global option to disable the fast path? There is one for
`uv build` because `--force-pep517` moves `uv build` much closer to a
`pip install` from source that a user of a library would experience (See
discussion at #9610), but uv overall doesn't really make guarantees
around the build env of dependencies, so I consider the direct build a
valid option.
Best reviewed commit-by-commit, only the last commit is the actual
implementation, while the preview mode introduction is just a
refactoring touching too many files.
When building a wheel from a source distribution or both a source
distribution and a wheel, the versions in their filenames must be the
same.
By inspecting the filenames, we also assert that the filenames from the
build a valid (we don't enforce normalization though, just that uv can
parse them).
Note that we're not yet checking that also the `pyproject.toml` version,
if declared, and METADATA version matches.
When running `lock_requires_python_exact`, we would download CPython
3.12.0 each time. By instead downloading CPython 3.13.0 ahead of time
and passing it in, we speed the test up and avoid timeouts. Locally in
pycharm, the test goes from 6.5s to 500ms.
When encountering `dynamic = ["version"]` in the pyproject.toml of a
source dist, we can ignore that and treat it as a statically known
metadata distribution, since the filename tells us the version and that
version must not change on build.
This fixed locking PyGObject 3.50.0 from `pygobject-3.50.0.tar.gz`
(minimized):
```toml
[project]
name = "PyGObject"
description = "Python bindings for GObject Introspection"
requires-python = ">=3.9, <4.0"
dependencies = [
"pycairo>=1.16"
]
dynamic = ["version"]
```
Afterwards, `uv add --no-sync toga` passes on Ubuntu 24.04 without the
pygobject build deps, when previously it needed `{ name = "pygobject",
version = "3.50.0", requires-dist = [], requires-python = ">=3.9" }`.
I've added a check that source distribution versions are respected after
build.
Fixes#9548
Add the `uv build --list`, a "subcommand" to list the files that would
be included when building a distribution. It does not build the
distribution, except when a source dist is required for source dist ->
wheel. This is an important debugging tool for the include and exclude
options: Did i actually include the files I wanted, or am i shipping a
broken distribution? Are there any temporary files I still need to
exclude?
Cargo offers this as `cargo package --list`.
`--list` is preview-exclusive, since it requires the fast path, which I
also put into preview.
Examples:
I'll fix the error handling in a follow-up.
Tagging as enhancement because it changes the stable output slightly
(two lines instead of one).
CC @charliermarsh for uv-wide consistency in the stdout/stderr handling.
## Summary
This change introduces the `UV_NO_INSTALLER_METADATA` environment
variable
as a way to opt out of the extra installer metadata files that `uv` is
creating.
This is important to achieve reproducible builds in distribution
packaging, allowing to replace usage of
[installer](https://pypi.org/project/installer) with `uv pip install`.
At the time of writing these files are:
- `uv_cache.json`
Contains timestamps which are non-reproducible.
These hashes also leak in to the `RECORD` file.
- `direct_url.json`
Contains the path to the installed wheel.
While not non-reproducible it's not required for distribution packaging.
- `INSTALLER`
Again, not non-reproducible, but of no value in distribution packaging.
## Test Plan
Automated test added.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Today, our dependency group implementation is a little awkward... For
each package `P`, we check if `P` contains dependencies for each enabled
group, then add a dependency on `P` with the group enabled. There are a
few issues here:
1. It's sort of backwards... We add a dependency from the base package
`P` to `P` with the group enabled. Then `P` with the group enabled adds
a dependency on the base package.
2. We can't, e.g., enable different groups for different packages. (We
don't have a way for users to specify this on the CLI, but there's no
reason that it should be _impossible_ in the resolver.)
3. It's inconsistent with how extras work, which leads to confusing
differences in the resolver.
Instead, our internal requirement type can now include dependency
groups, which makes dependency groups look much, much more like extras
in the resolver.
Using the directory writer trait, we can collect the files instead of
writing them to a real sink. This builds up to a `uv build --list`
similar to `cargo package --list`. It is not connected to the cli yet.
## Summary
Discovered while working on https://github.com/astral-sh/uv/issues/9516.
In the linked repo, the root uses a `../dependency` path for the
workspace member, which we weren't normalizing.
## Summary
If a Git repository uses a `path` dependency (rather than a
`workspace`), we need to expand the path to make it relative to the Git
root.
Closes https://github.com/astral-sh/uv/issues/9516.
## Summary
Include the `git_member` when fetching metadata from cache.
h/t to @PhilipVinc for the suggested fix
Resolves#8887
## Test Plan
Pending
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This pull request is best viewed with [whitespace
hidden](https://github.com/astral-sh/uv/pull/8650/files?diff=unified&w=1)
Adds a `--default` flag to `uv python install` in preview. This includes
a `python` and `python{major}` executable in addition to the
`python{major}.{minor}` executable. We will replace uv-managed
executables, but externally managed executables require the `--force`
flag to overwrite.
If you run `uv python install` (without arguments), we include the
`--default` flag implicitly to populate `python` and `python3` for the
"default" install version.
In the future, we should add a warning if the installed executable isn't
at the front of the PATH.
## Summary
Fixes#9027
Minor enhancement on top of #8531 that makes the CLI parameter
`--check-url` also available as the setting `check-url` in configuration
files.
## Test Plan
Updates existing tests to take the new setting into account.
Within publish command testing I didn't see existing tests covering
settings from toml files (instead of from CLI params), so I didn't add
anything of that sort.
## Summary
On Windows, non-virtual environments put the `python.exe` in the
top-level of the installation directory, rather than in `Scripts`. This
PR adds those paths to `PATH` in `uv run` and `uv tool run`.
Closes
https://github.com/astral-sh/uv/issues/9574#issuecomment-2512217110.
This _partially_ unwinds the optimization in #9540 by adding back the
base package dependency as a sibling to the extra package dependency
in some cases. Specifically, this occurs when _any_ of the extras are
declared as conflicting.
This is believed to be necessary (until another method is found) to
handle the forking logic based on conflicts. Namely, the forking logic
depends on the base and extra packages being sibling dependencies. If
only the extra is present, then it won't be included in the fork that
excludes all conflicting extras. And that means the base package won't
either, even though it should be included in that fork in some cases. If
the base package dependency is deferred, then it will never be reached.
This also adds another test and updates the snapshots that would have
caught the regression in #9540 if the conflict tests had been enabled.
Embarrassingly, PR #9474 moved the conflicting extras/groups tests into
their own module, but never actually included the module in
`it/main.rs`.
This adds `lock_conflict` to `main.rs` and fixes the fallout.
For listing files, we first use a directory writer for source dists,
which we will use for collecting the filenames instead of writing the
archive in the future. I've split breaking `lib.rs` of uv-build-backend
into modules into the next PR.
No logic changes, only restructuring.
Best reviewed commit-by-commit
Going through PEP 517 to build a package is slow, so when building a
package with the uv build backend, we can call into the uv build backend
directly. This is the basis for the `uv build --list`.
This does not enable the fast path for general source dependencies.
There is a possible difference in execution if the latest uv version is
newer than the one currently running: The PEP 517 path would use the
latest version, while the fast path uses the current version.
Please review commit-by-commit
### Benchmark
`built_with_uv`, using the fast path:
```
$ hyperfine "~/projects/uv/target/profiling/uv build"
Time (mean ± σ): 9.2 ms ± 1.1 ms [User: 4.6 ms, System: 4.6 ms]
Range (min … max): 6.4 ms … 12.7 ms 290 runs
```
`hatcling_editable`, with hatchling being optimized for fast startup
times:
```
$ hyperfine "~/projects/uv/target/profiling/uv build"
Time (mean ± σ): 270.5 ms ± 18.4 ms [User: 230.8 ms, System: 44.5 ms]
Range (min … max): 250.7 ms … 298.4 ms 10 runs
```
In the course of working on #9289, I've had to devise
some additions to our markers. While we are still staying
strictly compatible with the PEP 508 format, we will be
abusing the `extra` expression to carry a lot more
information.
Specifically, we want the following additional
operations:
* Simplify `extra != 'foo'`
* Remove all extra expressions
* Remove everything except extra expressions
My work on #9289 requires all of these (which will be
in a future in PR).
## Summary
This proposes adding the command line option `uv pip uninstall --dry-run
...`, complementing the existing `uv pip install --dry-run ...` added
for #1244 in #1436.
This option does not exist in PyPA's `pip uninstall`, if adopted it
would be unique to `uv pip`. The code should be considered PoC, it is
baby's first Rust.
The initial motivation was while investigating
https://github.com/moreati/ansible-uv/issues/2 - to allow Ansible module
`moreati.uv.pip` to work with`state: absent` in "check_mode" (Ansible's
equivalent of a dry run), without requiring `packaging` or `setuptools`.
## Test Plan
One new unit test has been added. I pedge to add more if the feature is
desired/accepted
Example usage
```console
➜ uv git:(pip-uninstall--dry-run) rm -rf .venv
➜ uv git:(pip-uninstall--dry-run) ./target/debug/uv venv
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
➜ uv git:(pip-uninstall--dry-run) ./target/debug/uv pip install httpx
Resolved 7 packages in 178ms
Prepared 5 packages in 60ms
Installed 7 packages in 15ms
+ anyio==4.6.2.post1
+ certifi==2024.8.30
+ h11==0.14.0
+ httpcore==1.0.7
+ httpx==0.28.0
+ idna==3.10
+ sniffio==1.3.1
➜ uv git:(pip-uninstall--dry-run) ./target/debug/uv pip uninstall --dry-run httpx
Would uninstall 1 package
- httpx==0.28.0
➜ uv git:(pip-uninstall--dry-run) ./target/debug/uv pip list
Package Version
-------- -----------
anyio 4.6.2.post1
certifi 2024.8.30
h11 0.14.0
httpcore 1.0.7
httpx 0.28.0
idna 3.10
sniffio 1.3.1
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Fixes#9531
## Context
While working with [uv](https://github.com/astral-sh/uv), I encountered
issues with a python dependency, [httpx](https://www.python-httpx.org/)
unable to be installed because of a **os error 5 permission denied**.
The error occur when we try to persist a **.exe file** from a temporary
folder into a persistent one.
I only reproduce the issue in an enterprise **Windows** Jenkins Runner.
In my virtual machines, I don't have any issues. So I think this is most
probably coming from the system configuration. This windows runner
**contains an AV/EDR**. And the fact that the file locked occured only
once for an executable make me think that it's most probably the cause.
While doing some research and speaking with some colleagues (hi
@vmeurisse), it seems that the issue is a very recurrent one on Windows.
In the Javascript ecosystem, there is this package, created by the
@isaacs, `npm` inventor: https://www.npmjs.com/package/graceful-fs, used
inside `npm`, allowing its package installations to be more resilient to
filesystem errors:
> The improvements are meant to normalize behavior across different
platforms and environments, and to make filesystem access more resilient
to errors.
One of its core feature is this one:
> On Windows, it retries renaming a file for up to one second if EACCESS
or EPERM error occurs, likely because antivirus software has locked the
directory.
So I tried to implement the same algorithm on `uv`, **and it fixed my
issue**! I can finally install `httpx`.
Then, [as I mentionned in this
issue](https://github.com/astral-sh/uv/issues/9531#issuecomment-2508981316),
I saw that you already implemented exactly the same algorithm in an
asynchronous function for renames 😄22fd9f7ff1/crates/uv-fs/src/lib.rs (L221)
## Summary of changes
- I added a similar function for `persist` (was not easy to have the
benediction of the borrow checker 😄)
- I added a `sync` variant of `rename_with_retry`
- I edited `install_script` to use the function including retries on
Windows
Let me know if I should change anything 🙂
Thanks!!
## Test Plan
This pull-request should be totally iso-functional, so I think it should
be covered by existing tests in case of regression.
All tests are still passing on my side.
Also, of course validated that my windows machines (windows 10 & windows
11) containing AV/EDR software are now able to install `httpx.exe`
script.
However, if you think any additional test is needed, feel free to tell
me!
When looking at the build frontend code, I noticed that we always pass
every single field of the shared state to the build dispatch:
```rust
let build_dispatch = BuildDispatch::new(
...
&state.index,
&state.git,
&state.capabilities,
&state.in_flight,
...
);
```
We can abstract this by moving `SharedState` into the build dispatch.
The `BuildDispatch` then has only immutable fields and the
`SharedState`. Since the `SharedState` is all `Arc`s, we can clone it
freely.
## Summary
After #9524, I noticed two other dependencies were misaligned.
Since the previous PR has been merged, I was thinking I could submit
those two misses.
Of course, open to any comments/decline!
Thanks!! 🙂
## Test Plan
All units tests are still passing on my side. Let's see with the
pull-request CI again 😄
## Summary
Previously, when we encountered `foo[bar]`, we'd add a dependency on
`PubGrubPackage::Package` for `foo`, and then `PubGrubPackage::Extra`
for `foo[bar]`.
Later, when we ask for the dependencies of the `PubGrubPackage::Extra`,
we add `PubGrubPackage::Package` for `foo`, and
`PubGrubPackage::Package` for `foo[bar]`. This is an intentional
strategy because it ensures that PubGrub "knows" that these have to be
solved to the same version as early as possible.
It turns out that the first part here ("add a dependency on
`PubGrubPackage::Package` for `foo`") is suboptimal, because it means
PubGrub might try to solve _just_ `foo` without realizing that it also
has to accommodate all the constraints from the extra.
Instead, we now add _just_ `PubGrubPackage::Extra` for `foo[bar]`, and
defer adding the base package. It looks like this leads to a far more
efficient solve for Airflow.
When adding excludes, we usually don't want to include python cache
files. On the contrary, I haven't seen any project in my ecosystem
research that would want any of `__pycache__`, `*.pyc`, `*.pyo` to be
included. By moving them behind a `default-excludes` toggle, they are
always active even when defining custom excludes, but can be deactivated
if the user so chooses.
With includes and excludes being this small again, we can roll back the
include-exclude anchored difference to always using anchored globs (i.e.
you would need to use `**/build-*.h` below).
A pyproject.toml with custom settings with the change applied:
```toml
[project]
name = "foo"
version = "0.1.0"
readme = "README.md"
license-files = ["LICENSE*", "third-party-licenses/*"]
[tool.uv.build-backend]
# A file we need for the source dist -> wheel step, but not in the wheel itself (currently unused)
source-include = ["data/build-script.py"]
# A temporary or generated file we want to ignore
source-exclude = ["/src/foo/not-packaged.txt"]
# Headers are build-only
wheel-exclude = ["build-*.h"]
[tool.uv.build-backend.data]
scripts = "scripts"
data = "assets"
headers = "header"
[build-system]
requires = ["uv>=0.5.5,<0.6"]
build-backend = "uv"
```
When building the source distribution, we always need to include
`pyproject.toml` and the module, when building the wheel, we always
include the module but nothing else at top level. Since we only allow a
single module per wheel, that means that there are no specific wheel
includes. This means we have source includes, source excludes, wheel
excludes, but no wheel includes: This is defined by the module root,
plus the metadata files and data directories separately.
Extra source dist includes are currently unused (they can't end up in
the wheel currently), but it makes sense to model them here, they will
be needed for any sort of procedural build step.
This results in the following fields being relevant for inclusions and
exclusion:
* `pyproject.toml` (always included in the source dist)
* project.readme: PEP 621
* project.license-files: PEP 639
* module_root: `Path`
* source_include: `Vec<Glob>`
* source_exclude: `Vec<Glob>`
* wheel_exclude: `Vec<Glob>`
* data: `Map<KnownDataName, Path>`
An opinionated choice is that that wheel excludes always contain the
source excludes: Otherwise you could have a path A in the source tree
that gets included when building the wheel directly from the source
tree, but not when going through the source dist as intermediary,
because A is in source excludes, but not in the wheel excludes. This has
been a source of errors previously.
In the process, I fixed a bug where we would skip directories and only
include the files and were missing license due to absolute globs.
## Summary
When we serialize and deserialize the lockfile, we remove the conflict
markers. So in the linked case, the edges for the `tqdm` entries are
like:
```
complexified_marker: UniversalMarker {
pep508_marker: python_full_version >= '3.9.0',
conflict_marker: true,
},
```
However... when we evaluate in-memory, the conflict markers are still
there...
```
complexified_marker: UniversalMarker {
pep508_marker: true,
conflict_marker: extra == 't1' and extra != 't2',
},
```
So if `uv run` creates the lockfile, we evaluate this as `false`.
We should make this consistent, and I expect @BurntSushi is aware. But
for now, it's reasonable / correct to pass the extra when evaluating at
this specific point, since we know the dependency was enabled by the
marker.
Closes
https://github.com/astral-sh/uv/issues/9533#issuecomment-2508908591.
When changing something about the settings,
`invalid_pyproject_toml_option_unknown_field` would fail unexpectedly
because the exact list of possible options had changed. Since we're
already testing this list in the settings-related test
`resolve_config_file`, i'm stubbing the exact output here.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
While working on potential bug fixes with temporary files on Windows (I
think I am currently ecountering the same issue as #2810)
I noticed that sub-workspaces were not all having the same `tempfile`
version. And they were not relying on the cargo root project dependency.
I don't know at all if it was done on purpose or not.
(I also wanted to override the root dependency with a local source but
it was not possible due to sub-workspaces not relying on the same).
The root lockfile already pinned to the `3.14.0`. Some sub-workspaces
were depending on the `3.12.0`, some others on the `3.14.0`. So I
updated the root `Cargo.toml` to the `3.14.0`.
Feel free to decline if it was done on purpose! No worries at all
🙂
Thanks!
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
All units tests are still passing on my side. Let's see with the
pull-request CI 😄
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
## Summary
When you pass a system drive to `Path::join`, Rust doesn't insert a
backslash between the drive and the path itself, so our lookups for
system configuration were failing.
Closes https://github.com/astral-sh/uv/issues/9416.
When trying to upload without a password but with the keyring, check
that the keyring has a password for the upload URL and username and warn
if it doesn't.
Fixes#8781
There are already a fair number and I'm planning to add more. And
`lock.rs` is already quite big.
There aren't any new tests or other changes here. This is just moving
tests and trimming down the function names to avoid redundancy in the
names.
## Summary
With `uv pip install --target` and `--prefix`, we (1) should allow
managed Pythons, and (2) should show a different message that's focused
on the interpreter we selected, rather than the environment.
## Summary
We still only respect overrides and constraints in the workspace root --
which we may want to change -- but overrides and constraints are now
correctly lowered.
Closes https://github.com/astral-sh/uv/issues/8148.
We were previously not uploading all metadata in the formdata of an
upload request in the legacy api. Notably, we were missing the PEP 639
license-files field.
I had to switch to pdm due to https://github.com/pypa/hatch/issues/1828
When performing a noop sync, we don't need the rayon threadpool, yet we
pay for its initialization:
Be making the initialization lazy, we avoid that cost:
This code runs every time before user code in `uv run`.
This means that before calling rayon, one now needs to call
`LazyLock::force(&RAYON_INITIALIZE);`.
Performance mode (CPU 0 is a perf core):
```
$ taskset -c 0 hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
Time (mean ± σ): 4.5 ms ± 0.1 ms [User: 2.7 ms, System: 1.8 ms]
Range (min … max): 4.4 ms … 6.4 ms 640 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
Time (mean ± σ): 4.4 ms ± 0.1 ms [User: 2.7 ms, System: 1.6 ms]
Range (min … max): 4.3 ms … 5.0 ms 679 runs
Summary
/home/konsti/projects/uv/target/profiling/uv sync ran
1.03 ± 0.04 times faster than /home/konsti/projects/uv/uv-main sync
```
Power saver mode:
```
$ hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
Time (mean ± σ): 28.1 ms ± 1.2 ms [User: 15.5 ms, System: 20.3 ms]
Range (min … max): 25.7 ms … 31.9 ms 102 runs
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
Time (mean ± σ): 24.0 ms ± 1.2 ms [User: 13.8 ms, System: 9.9 ms]
Range (min … max): 22.2 ms … 28.2 ms 122 runs
Summary
/home/konsti/projects/uv/target/profiling/uv sync ran
1.17 ± 0.08 times faster than /home/konsti/projects/uv/uv-main sync
```
## Summary
We never construct these -- they should be impossible, since we always
translate to `python_full_version`. This PR encodes that impossibility
in the types.
## Summary
This PR modifies our lowered representation such that any deprecated
aliases are treated as "the same" marker in the algebra.
So, for example, we now recognize that this is impossible, despite the
marker names being different:
```
typing-extensions ; platform.python_implementation == 'CPython' and python_implementation != 'CPython'
```
Similarly, we now recognize that this is just `sys_platform == 'win32'`,
despite the presence of both markers:
```
anyio ; sys_platform == 'win32' and sys.platform == 'win32'
```
## Summary
I want to move towards a more normalized marker representation within
the marker tree, which means that the things we warn against will
disappear by the time we get to evaluation. I think it makes more sense
to show these warnings when we create the tree, rather than when we
evaluate it.
As discussed in https://github.com/astral-sh/uv/issues/9423, it's
confusing that we do not allow `uv sync` just because the `.venv`
directory _exists_. This change matches `uv venv`.
<!--
Thank you for contributing to uv! To help us review effectively, please
ensure that:
- The pull request includes a summary of the change.
- The title is descriptive and concise.
- Relevant issues are referenced where applicable.
-->
## Summary
Resolves#9333
This pull request introduces support for the `--no-extra` command-line
flag and the corresponding `no-extra` UV setting.
### Behavior
- When `--all-extras` is supplied, the specified extras in `--no-extra`
will be excluded from the installation.
- If `--all-extras` is not supplied, `--no-extra` has no effect and is
safely ignored.
## Test Plan
Since `ExtrasSpecification::from_args` and
`ExtrasSpecification::extra_names` are the most important parts in the
implementation, I added the following tests in the
`uv-configuration/src/extras.rs` module:
- **`test_no_extra_full`**: Verifies behavior when `no_extra` includes
the entire list of extras.
- **`test_no_extra_partial`**: Tests partial exclusion, ensuring only
specified extras are excluded.
- **`test_no_extra_empty`**: Confirms that no extras are excluded if
`no_extra` is empty.
- **`test_no_extra_excessive`**: Ensures the implementation ignores
`no_extra` values that don't match any available extras.
- **`test_no_extra_without_all_extras`**: Validates that `no_extra` has
no effect when `--all-extras` is not supplied.
- **`test_no_extra_without_package_extras`**: Confirms correct behavior
when no extras are available in the package.
- **`test_no_extra_duplicates`**: Verifies that duplicate entries in
`pkg_extras` or `no_extra` do not cause errors.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This adds a `--prune` flag to the `export` command to correspond with
the `--prune` flag of the `tree` command.
The purpose is for generating a `requirements.txt` that omits a package
and all of that package's unique dependencies. This is useful for cases
where the project has a dependency on a common core package, but where
that package does not need to be installed in the target environment.
For example, a pyspark job needs spark for development, but when
installing into a cluster that already has pyspark installed, it is
desirable to omit pyspark's whole dependency tree so that only the
unique dependencies that your job needs get installed, and do not risk
breaking the pyspark dependencies with something incompatible.
Dev groups cannot always cover this case because there are other
projects where this common dependency occurs as a transitive. One
example is Airflow providers, which include Airflow itself as a
dependency, but it is unnecessary and undesirable to include Airflow's
dependency tree in the `requirements.txt` for your DAGs.
Partly related to #7214, though I'm not sure it covers the ask in that
one of having this functionality extend to the project's actual
published metadata.
## Test Plan
An integration test was added, and some manual testing. Let me know if
more would be better.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This change is correct because disjointness checks now
incorporate conflicts. In this case, there are actually
four forks. Two of them correspond to
`sys_platform == 'darwin'` and `sys_platform != 'darwin'`,
but neither of those contain `jinja2==3.1.3`. Instead,
they contain other versions of `jinja2` linked to other
extras.
If we ever add conflicts to our `resolution-markers` in
the lock file, then those forks should show up here
again. (Because, of course, some forks do contain
`jinja2==3.1.3` here.)
When we generate conflict markers for each resolution after the
resolver runs, it turns out that generating them just from exclusion
rules is not sufficient.
For example, if `foo` and `bar` are declared as conflicting extras, then
we end up with the following forks:
A: extra != 'foo'
B: extra != 'bar'
C: extra != 'foo' and extra != 'bar'
Now let's take an example where these forks don't share the same version
for all packages. Consider a case where `idna==3.9` is in forks A and C,
but where `idna==3.10` is in fork B. If we combine the markers in forks
A and C through disjunction, we get the following:
idna==3.9: extra != 'foo' or (extra != 'foo' and extra != 'bar')
idna==3.10: extra != 'bar'
Which simplifies to:
idna==3.9: extra != 'foo'
idna==3.10: extra != 'bar'
But these are clearly not disjoint. Both dependencies could be selected,
for example, when neither `foo` nor `bar` are active. We can remedy this
by keeping around the inclusion rules for each fork:
A: extra != 'foo' and extra == 'bar'
B: extra != 'bar' and extra == 'foo'
C: extra != 'foo' and extra != 'bar'
And so for `idna`, we have:
idna==3.9: (extra != 'foo' and extra == 'bar') or (extra != 'foo' and extra != 'bar')
idna==3.10: extra != 'bar' and extra == 'foo'
Which simplifies to:
idna==3.9: extra != 'foo'
idna==3.10: extra != 'bar' and extra == 'foo'
And these *are* properly disjoint. There is no way for them both to be
active. This also correctly accounts for fork C where neither `foo` nor
`bar` are active, and yet, `idna==3.9` is still enabled but `idna==3.10`
is not. (In the [motivating example], this comes from `baz` being enabled.)
That is, this captures the idea that for `idna==3.10` to be installed,
there must actually be a specific extra that is enabled. That's what
makes it disjoint from `idna==3.9`.
We aren't quite done yet, because this does add *too many* conflict
markers to dependency edges that don't need it. In the next commit,
we'll add in our world knowledge to simplify these conflict markers.
[motivating example]: https://github.com/astral-sh/uv/issues/9289
I think Ibraheem had this routine at some point in the past, but
we ended up dropping it because we didn't have a use for it. Well,
now we do!
It turns out that when we generate "conflict markers," they don't
actually take "world knowledge" into account. In particular, there
is "world knowledge" that a particular set of extras cannot be
enabled simultaneously. This in turn allows us to simplify most
conflict markers. If we didn't do this, it's likely that lock files
would become littered with conflict markers whenever any conflicts
are declared.
This is somewhat (although not completely) analogous to how we
"simplify" markers with respect to `requires-python`. That is,
`requires-python` reflects world knowledge that enables markers
to be written more simply than they otherwise would be without
world knowledge.
Previously, we had copied the behavior of `try_markers` to return
`None` in the case where the marker was always true. I believe this
was done because it somewhat implies that there is no forking
happening. But I find this somewhat strange personally, and instead
flipped this around so that it still returns a marker in that case.
The one call site that is impacted by this is the resolution
graph construction. If we left it as-is, it would end up with
a list of one marker that is always true in some cases. And this
in turn results in writing an empty `resolution-markers` to the
lock file. Probably the output logic should be tweaked instead,
but we leave it alone for now.
## Summary
If we're installing with `--target` or `--prefix`, then it's not a
mutable operation, so we should be allowed to discover system Pythons. I
suspect this was hard to special-case in the past but is now trivial
after @zanieb's various refactors.
Closes https://github.com/astral-sh/uv/issues/9356.
## Summary
Aligns the description of `UV_NO_PROGRESS` with other env vars that also
have a related flag.
`--no-progress` is a "global option" and exists in every command.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This effectively combines a PEP 508 marker and an as-yet-specified
marker for expressing conflicts among extras and groups.
This just defines the type and threads it through most of the various
points in the code that previously used `MarkerTree` only. Some parts
do still continue to use `MarkerTree` specifically, e.g., when dealing
with non-universal resolution or exporting to `requirements.txt`.
This doesn't change any behavior.
This doesn't change any behavior. But this makes it a bit
clearer in the code that `uv pip compile` does not support
specifying conflicts. Indeed, we always pass an empty set of
conflicts to the resolver.
This is because there is no way to encode the conditional
logic of conflicts into the `requirements.txt` format. This
is unlike markers.
This doesn't change any behavior. My guess is that this code was
a casualty of refactoring. But basically, it was doing redundant
case analysis and iterating over all resolutions (even though it's
in the branch that can only occur when there is only one
resolution).
This filtering is now redundant, since forking now avoids these
degenerate cases by construction.
The main change to forking that enables skipping over "always
false" forks is that forking now starts with the parent's markers
instead of starting with MarkerTree::TRUE and trying to combine
them with the parent's markers later. This in turn leads to
skipping over anything that "can't" happen when combined with the
parents markers. So we never hit the case of generating a fork
that, when combined with the parent's markers, results in a
marker that is always false. We just avoid it in the first place.
This test demonstrates the difference between `extra != "foo"` and
`sys_platform != "foo"`.
I wrote this test down to test the extra simplification logic was
correct. And I also wanted to test whether we could somehow hackily
encode `group` (as opposed to just `extra`) logic into marker
expressions by reusing another field. But I don't think we can.
_get_glibc_version() can after #9005 return either (0, 0) if glibc
string is missing or (-1, -1) if the string can't be parsed. There was
no need to change missing string to (0, 0).
Also, move back indentation to make it easier to understand.
Currently, user display returns an empty path if the current dir is the
directory we are printing. This leads to odd messages such as
> Including project.license-files at `` with `LICENSE*`
or
> Not a license files match: ``
Instead, we display the current path as a dot.
## Summary
`--upgrade` isn't useful, since it's the default. So it's now hidden,
but continues to warn if you enable it.
`--no-upgrade` isn't useful, since it panics. So it's now removed
entirely. This isn't breaking, since it already didn't work.
`--upgrade-package` actually _is_ useful, because it turns out it allows
things like: `uv tool upgrade babel --upgrade-package "babel<0.2.14"` to
constrain the upgrade.
I left this in place but hid it... I think we should provide a better
workflow for this, like `uv tool upgrade "babel<0.2.14"`? It's strange
to specify the package twice, and that `uv tool upgrade` has an
`--upgrade-package` flag.
Closes https://github.com/astral-sh/uv/issues/9317.
## Summary
On Termux, uv currently fails to find any interpreter because it can't
find a glibc version, because there isn't one. But the Python
interpreter is still functional nonetheless.
So, when glibc cannot be found, simply return 0 for the version numbers
and mark the interpreter as being incompatible with manylinux
I really don't know if this is the right way to address this, but I can
attest that manual testing shows uv appears to be fully functional, at
least for pip and virtualenvs.
Fixes#7373
## Test Plan
I tried running the test suite, and after some tweaks, a good portion of
the test suite passes as well. A significant number of tests fail, but
this appears to be due to minor differences in output, like warnings
about hard links not working (hard links are completely disallowed on
Android), differences in the number of files removed, etc. The test
suite seems to be very sensitive to minor variations in output.
This PR contains three smaller improvements:
* Improve the include/exclude logging. We're still showing the current
directory as empty backticks, not sure what to do about that
* Add early stopping to license file globbing, so we don't traverse the
whole directory recursively when license files can only be in few places
* Support explicit wheel excludes. These are still not entirely right,
but at least we're correctly excluding compiled python files now. The
next step is to make sure that the wheel excludes contain all pattern
from source dist excludes, to make sure source tree -> wheel can't have
more files than source tree -> source dist -> wheel.
## Summary
The issue here is fairly complex. Consider the following:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12.0"
dependencies = []
[project.optional-dependencies]
cpu = [
"torch>=2.5.1",
"torchvision>=0.20.1",
]
cu124 = [
"torch>=2.5.1",
"torchvision>=0.20.1",
]
[tool.uv]
conflicts = [
[
{ extra = "cpu" },
{ extra = "cu124" },
],
]
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
]
torchvision = [
{ index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
```
When solving this project, we first pick a PyTorch version from PyPI, to
solve the `cu124` extra, selecting `2.5.1`.
Later, we try to solve the `cpu` extra. In solving that extra, we look
at the PyTorch CPU index. Ideally, we'd select `2.5.1+cpu`... But
`2.5.1` is already a preference. So we choose that.
Now, we only respect preferences for explicit indexes if they came from
the same index.
Closes https://github.com/astral-sh/uv/issues/9295.
## Summary
The new `--index` and `--default-index` flags are being omitted in the
`uv pip compile` header, unintentionally.
Closes https://github.com/astral-sh/uv/issues/9287.
## Summary
I find myself messing this up with `--build-constraint` vs.
`--build-constraints`, and it turns out our own CLI isn't fully
consistent here either.
When building only a single crate in the workspace to run its tests, we
often recompile a lot of other, unrelated crates. Whenever cargo has a
different set of crate features, it needs to recompile. By moving some
features (non-exhaustive for now) to the workspace level, we always
activate them an avoid recompiling.
The cargo docs mismatch the behavior of cargo around default-deps, so I
filed that upstream and left most `default-features` mismatches:
https://github.com/rust-lang/cargo/issues/14841.
Reference script:
```python
import tomllib
from collections import defaultdict
from pathlib import Path
uv = Path("/home/konsti/projects/uv")
skip_list = ["uv-trampoline", "uv-dev", "uv-performance-flate2-backend", "uv-performance-memory-allocator"]
root_feature_map = defaultdict(set)
root_default_features = defaultdict(bool)
cargo_toml = tomllib.loads(uv.joinpath("Cargo.toml").read_text())
for dep, declaration in cargo_toml["workspace"]["dependencies"].items():
root_default_features[dep] = root_default_features[dep] or declaration.get("default-features", True)
root_feature_map[dep].update(declaration.get("features", []))
feature_map = defaultdict(set)
default_features = defaultdict(bool)
for crate in uv.joinpath("crates").iterdir():
if crate.name in skip_list:
continue
if not crate.joinpath("Cargo.toml").is_file():
continue
cargo_toml = tomllib.loads(crate.joinpath("Cargo.toml").read_text())
for dep, declaration in cargo_toml.get("dependencies", {}).items():
# If any item uses default features, they are used everywhere
default_features[dep] = default_features[dep] or declaration.get("default-features", True)
feature_map[dep].update(declaration.get("features", []))
for dep, features in sorted(feature_map.items()):
features = features - root_feature_map.get(dep, set())
if not features and default_features[dep] == root_default_features[dep]:
continue
print(dep, default_features[dep], sorted(features))
```
## Summary
Just trying to unify the retry handling, as in
https://github.com/astral-sh/uv/pull/9274 and elsewhere. Right now, the
publish handler doesn't use any backoff and always retries three times
regardless of settings.
## Summary
This was an oversight in the initial implementation. We shouldn't
validate sources for the `build-system.requires` field, since extras and
groups can _never_ be active.
Closes https://github.com/astral-sh/uv/issues/9259.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
In uv-globfilter, use the workspace `fs-err` in `dev-dependencies`.
This fixes an unnecessary dev-dependency on `fs-err` 2.x even after the
workspace fs-err was updated to 3.x in
https://github.com/astral-sh/uv/pull/8625.
The `Cargo.lock` file still has `fs-err v2.11.0` after this PR, but it
is via `tracing-durations-export v0.3.0` rather than directly required
by any `uv` crate.
## Test Plan
<!-- How was it tested? -->
```
$ cd crates/uv-globfilter/
$ cargo test
```
## Summary
It turns out that `WrappedReqwestError` skips the `reqwest::Error`
itself in order to hack the display. This PR adds it to the list of
variants we check when retrying transient errors.
Closes https://github.com/astral-sh/uv/issues/9246.
## Test Plan
Patched `reqwest` locally to return an error in `bytes()`. Verified that
it was _not_ caught prior to this PR, but was caught afterwards.
- Adds a collapsible section for the project concept
- Splits the project concept document into several child documents.
- Moves the workspace and dependencies documents to under the project
section
- Adds a mkdocs plugin for redirects, so links to the moved documents
still work
I attempted to make the minimum required changes to the contents of the
documents here. There is a lot of room for improvement on the content of
each new child document. For review purposes, I want to do that work
separately. I'd prefer if the review focused on this structure and idea
rather than the content of the files.
I expect to do this to other documentation pages that would otherwise be
very nested.
The project concept landing page and nav (collapsed by default) looks
like this now:
<img width="1507" alt="Screenshot 2024-11-14 at 11 28 45 AM"
src="https://github.com/user-attachments/assets/88288b09-8463-49d4-84ba-ee27144b62a5">
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
PR #4965 added `*-manylinux_2_31` as a target triple, and issue #4966
described the need for a more general solution.
In lieu of a general solution, this PR adds further explicit manylinux
target triples for different glibc version up to the one used by the
latest Ubuntu release (glibc 2.40 used in Ubuntu 24.10).
## Test Plan
<!-- How was it tested? -->
Local, manual testing with a Python wheel targeting
`x86_64-manylinux_2_35`.
Allow including data files in wheels, configured through
`pyproject.toml`. This configuration is currently only read in the build
backend. We'd only start using it in the frontend when we're adding a
fast path.
Each data entry is a directory, whose contents are copied to the
matching directory in the wheel in
`<name>-<version>.data/(purelib|platlib|headers|scripts|data)`. Upon
installation, this data is moved to its target location, as defined by
<https://docs.python.org/3.12/library/sysconfig.html#installation-paths>:
- `data`: Installed over the virtualenv environment root. Warning: This
may override existing files!
- `scripts`: Installed to the directory for executables, `<venv>/bin` on
Unix or `<venv>\Scripts` on Windows. This directory is added to PATH
when the virtual environment is activated or when using `uv run`, so
this data type can be used to install additional binaries. Consider
using `project.scripts` instead for starting Python code.
- `headers`: Installed to the include directory, where compilers
building Python packages with this package as built requirement will
search for header files.
- `purelib` and `platlib`: Installed to the `site-packages` directory.
It is not recommended to uses these two options.
For simplicity, for now we're just defining a directory to be copied for
each data directory, while using the glob based include mechanism in the
background. We thereby introduce a third mechanism next to the main
includes and the PEP 639 mechanism, which is not what we should finalize
on.
## Summary
The reqwest middleware doesn't retry errors that occur "after" the
request completes -- but in some cases, these do include spurious errors
that we want to retry. See https://github.com/astral-sh/uv/issues/8144
for examples. This PR adds a second retry layer during the response
_handler_, which should help with some of the spurious failures we see
in the linked issue.
Closes https://github.com/astral-sh/uv/issues/8144.
## Summary
We missed the case in which the user has a legacy non-`[project]` root
-- we were always installing all members.
Closes https://github.com/astral-sh/uv/issues/9214.
## Summary
This PR enables something like the "final boss" of PyTorch setups --
explicit support for CPU vs. GPU-enabled variants via extras:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = []
[project.optional-dependencies]
cpu = [
"torch==2.5.1+cpu",
]
gpu = [
"torch==2.5.1",
]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
[tool.uv]
conflicts = [
[
{ extra = "cpu" },
{ extra = "gpu" },
],
]
```
It builds atop the conflicting extras work to allow sources to be marked
as specific to a dedicated extra being enabled or disabled.
As part of this work, sources now have an `extra` field. If a source has
an `extra`, it means that the source is only applied to the requirement
when defined within that optional group. For example, `{ index =
"torch-cpu", extra = "cpu" }` above only applies to
`"torch==2.5.1+cpu"`.
The `extra` field does _not_ mean that the source is "enabled" when the
extra is activated. For example, this wouldn't work:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = ["torch"]
[tool.uv.sources]
torch = [
{ index = "torch-cpu", extra = "cpu" },
{ index = "torch-gpu", extra = "gpu" },
]
[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
```
In this case, the sources would effectively be ignored. Extras are
really confusing... but I think this is correct? We don't want enabling
or disabling extras to affect resolution information that's _outside_ of
the relevant optional group.
## Summary
These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
## Summary
I moved this to a separate test. The packages may or may not be
downloaded already, since the previous command fails -- it just depends
on timing.
## Summary
The distributions used to be stored in a `BTreeMap`, keyed by name.
They're now stored in a graph... so iteration isn't guaranteed to
produce a deterministic hash!
This fixes a "flaky" test, though it's actually a real bug. The test was
right!
Closes#9137.
Fixes#9164
Using clap's `default_value_t` makes the `flag` function unhappy, so
just set the default when we unwrap. Tested with no flags,
`--verify-hashes`, `--no-verify-hashes` and setting in uv.toml
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
When building source distributions, we need to include the readme, so it
can become part the METADATA body when building the wheel. We also need
to support the license files from PEP 639. When building the source
distribution, we copy those file relative to their origin, when building
the wheel, we copy them to `.dist-info/licenses`.
The test for idempotence in wheel building is merged into the file
listing test, which also covers that source tree -> source dist -> wheel
is equivalent to source tree -> wheel, though we do need to check for
file inclusion stronger here.
Best reviewed commit-by-commit
## Summary
Align uv's workspace discovery with red knots (see
https://github.com/astral-sh/ruff/pull/14308#issuecomment-2474296308)
* Detect nested workspace inside the current workspace rather than
testing if the current workspace is a member of any outer workspace.
* Detect packages with identical names.
## Test Plan
I added two integration tests. I also back ported the tests to main to
verify that both these invalid workspaces aren't catched by uv today.
That makes this, technically, a breaking change but I would consider the
change a bug fix.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
I was wrongly using `.name()` to detect if a package was "not root", but
in `pip compile`, the root can have a name -- so we were failing to find
the derivation chain.
## Summary
This PR adds context to our error messages to explain _why_ a given
package was included, if we fail to download or build it.
It's quite a large change, but it motivated some good refactors and
improvements along the way.
Closes https://github.com/astral-sh/uv/issues/8962.
## Summary
This PR should not contain any user-visible changes, but the goal is to
refactor the `Resolution` type to retain a dependency graph. We want to
be able to explain _why_ a given package was excluded on error (see:
https://github.com/astral-sh/uv/issues/8962), which in turn requires
that at install time, we can go back and figure out the dependency
chain. At present, `Resolution` is just a map from package name to
distribution; this PR remodels it as a graph in which each node is a
package, and the edges contain markers plus extras or dependency groups.
A first milestone: source tree -> source dist -> wheel -> install works.
This PR adds a test for this.
There's obviously a lot still missing, including basics such as the
Readme inclusion.
## Summary
As discussed in Discord... This struct has evolved to include a lot of
information apart from the `petgraph::Graph`. And I want to add a graph
to the simplified `Resolution` type. So I think this name makes more
sense.
When doing a directory traversal for source dist inclusion, we want to
offer the user include and exclude options, and we want to avoid
traversing irrelevant directories. The latter is important for
performance, especially on network file systems, but also with large
data directories, or (not-included) directories with other permissions.
To support this, we introduce `GlobDirFilter`, which uses a DFA from
regex_automata to determine whether any children of a directory can be
included and skips the directory if not.
The globs are based on PEP 639. The syntax is more restricted than glob
or globset, but it's standardized. I chose it over glob or globset
because we're already using this syntax for `project.license-files` a
required by PEP 639, so it makes sense to use the same globs for all
includes (see e.g.
4f52a3bb62/pyproject.toml (L36-L48)
for example with same semantics for include and exclude)
### Semantics
Glob semantics are complex due to mixing directories and files,
expectations around simplicity and our need to exclude most of the tree
in the project from traversal. The current draft uses a syntax that
optimizes for simple default use cases for the start.
#### includes
Glob expressions which files and directories to include in the source
distribution.
Includes are anchored, which means that `pyproject.toml` includes only
`<project root>/pyproject.toml`. Use for example `assets/**/sample.csv`
to include for all
`sample.csv` files in `<project root>/assets` or any child directory. To
recursively include
all files under a directory, use a `/**` suffix, e.g. `src/**`. For
performance and
reproducibility, avoid unanchored matches such as `**/sample.csv`.
The glob syntax is the reduced portable glob from
[PEP 639](https://peps.python.org/pep-0639/#add-license-FILES-key).
#### excludes
Glob expressions which files and directories to exclude from the
previous source
distribution includes.
Excludes are not, which means that `__pycache__` excludes all
directories named
`__pycache__` and it's children anywhere. To anchor a directory, use a
`/` prefix, e.g.,
`/dist` will exclude only `<project root>/dist`.
The glob syntax is the reduced portable glob from
[PEP 639](https://peps.python.org/pep-0639/#add-license-FILES-key).
Surprisingly, this seems to be all that's necessary.
Previously, we were only extracting an extra from a
PubGrubPackage to test for conflicts. But now we extract
either an extra or a group. The surrounding code all
remains the same.
We do need to add some extra checking for groups
specifically, but I believe that's it.
This adds support for providing conflicting group names in addition to
extra names to `Conflicts`.
This merely makes "room" for it in the types while keeping everything
working. We'll add proper support for it in the next commit.
Note that one interesting trick we do here is depend directly on
`hashbrown` so that we can make use of its `Equivalent` trait. This in
turn lets us use things like `ConflictItemRef` as a lookup key for a
hashset that contains `ConflictItem`. This mirrors using a `&str` as a
lookup key for a hashset that contains `String`, but works for arbitrary
types. `std` doesn't support this, but `hashbrown` does. This trick in
turn lets us simplify some of our data structures.
This also rejiggers some of the serde-interaction with the conflicting
types. We now use a wire type to represent our conflicting items for
more flexibility. i.e., Support `extra` XOR `group` fields.
Since this is intended to support _both_ groups and extras, it doesn't
make sense to just name it for groups. And since there isn't really a
word that encapsulates both "extra" and "group," we just fall back to
the super general "conflicts."
We'll rename the variables and other things in the next commit.
## Summary
I need this for the derivation chain work
(https://github.com/astral-sh/uv/issues/8962), but it just seems
generally useful. You can't always get a version from a `Dist` (it could
be URL-based!), but when we create a `ResolvedDist`, we _do_ know the
version (and not just the URL). This PR preserves it.
## Summary
This PR ensures that `pylint>=3.2.6` followed by
`pylint-module-boundaries>=1.3.1` is considered sorted, despite the fact
that `>` is later in the alphabetic than `-`. By purely comparing
strings, they would _not_ be sorted; but by considering the name, then
the specifiers, they are.
Closes https://github.com/astral-sh/uv/issues/9076.
## Summary
Part of me wants to revert support for `--with "flask, requests"`, but
the multiple specifiers case actually isn't ambiguous, and handling it
is better than shipping a breaking change in a patch release.
Closes https://github.com/astral-sh/uv/issues/9081.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
-->
## Summary
Adds python-install-mirror and pypy-install-mirror as keys for uv.toml,
and cli args for `uv python install`.
Could leave the cli args out if we think the env vars and configs are
sufficient.
Fixes#8186
<!-- What's the purpose of the change? What does it do, and why? -->
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This restores behavior previously removed in
https://github.com/astral-sh/uv/pull/7649.
I thought it'd be clearer (and simpler) to have a consistent Python
executable name ordering. However, we've seen some cases where this can
be surprising and, in combination with #8481, can result in incorrect
behavior. For example, see https://github.com/astral-sh/uv/issues/9046
where we prefer `python3` over `python3.12` in the same directory even
though `python3.12` was requested. While `python3` and `python3.12` both
point to valid Python 3.12 environments there, the expectation is that
when `python3.12` is requested that the `python3.12` executable is
preferred. This expectation may be less obvious if the user requests
`python@3.12`, but uv does not distinguish between these request forms.
Similarly, this may be surprising as by default uv prefers `python` over
`python3` but when requesting `python3.12` the preference will be
swapped.
This PR adds support for conflicting extras. For example, consider
some optional dependencies like this:
```toml
[project.optional-dependencies]
project1 = ["numpy==1.26.3"]
project2 = ["numpy==1.26.4"]
```
These dependency specifications are not compatible with one another.
And if you ask uv to lock these, you'll get an unresolvable error.
With this PR, you can now add this to your `pyproject.toml` to get
around this:
```toml
[tool.uv]
conflicting-groups = [
[
{ package = "project", extra = "project1" },
{ package = "project", extra = "project2" },
],
]
```
This will make the universal resolver create additional forks
internally that keep the dependencies from the `project1` and
`project2` extras separate. And we make all of this work by reporting
an error at **install** time if one tries to install with two or more
extras that have been declared as conflicting. (If we didn't do this,
it would be possible to try and install two different versions of the
same package into the same environment.)
This PR does *not* add support for conflicting **groups**, but it is
intended to add support in a follow-up PR.
Closes#6981Fixes#8024
Ref #6729, Ref #6830
This should also hopefully unblock
https://github.com/dagster-io/dagster/pull/23814, but in my testing, I
did run into other problems (specifically, with `pywin`). But it does
resolve the problem with incompatible dependencies in two different
extras once you declare `test-airflow-1` and `test-airflow-2` as
conflicting for `dagster-airflow`.
NOTE: This PR doesn't make `conflicting-groups` public yet. And in a
follow-up PR, I plan to switch the name to `conflicts` instead of
`conflicting-groups`, since it will be able to accept conflicting extras
_and_ conflicting groups.
`uv init` shouldn't have been using `EnvironmentPreference::Any` for
discovery of a Python interpreter, it seems like an oversight that it
was reading from virtual environments. I changed it to
`EnvironmentPreference::OnlySystem` so we'll use the first Python on the
`PATH` instead. However, I think we actually do want to respect a
virtual environment's Python version if it's in the target project
directory already, so I've implemented that as well.
Closes https://github.com/astral-sh/uv/issues/9072
Closes https://github.com/astral-sh/uv/issues/8092
## Summary
Not thrilled with this but helps for now. I feel like this
error-handling should happen at the top-level, rather than on all these
individual commands. But we don't have a unified result type at the
top-level of the CLI -- all these commands return `anyhow::Result`.
## Summary
Shows similar diagnostics for failures that happen at install time,
rather than resolve time. This will ultimately feed into
https://github.com/astral-sh/uv/issues/8962 since we'll now have
consolidated handling for these kinds of failures.
## Summary
If a `uv add` fails at the sync stage, we need to clean up the changes
to the `uv.lock`, since it might've been edited during in the lock stage
(which, by necessity, succeeded). As-is, we revert the `pyproject.toml`
but not the `uv.lock`, so the two are out-of-sync.
Closes https://github.com/astral-sh/uv/issues/9028.
Closes https://github.com/astral-sh/uv/issues/7992.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR builds off of https://github.com/astral-sh/uv/pull/6738 to fix
#6724 (sorry for the new PR @charliermarsh I didn't want to push to your
branch, not even sure if I could). The reason the original PR doesn't
fix the issue described in #6724 is because the fastapi is ran in the
project context (as I assume a lot of use cases are). This PR adds an
extra commit to handle the signals in the project/run.rs file
~It also addresses the comment
[here](https://github.com/astral-sh/uv/pull/6738/files#r1734757548) to
not use the tokio ctrl-c method since we are now handling SIGINT
ourselves~ update, tokio handles SIGINT in a platform agnostic way,
intercepting this ouselves makes the logic more complicated with
windows, decided to leave the tokio ctrl-c handler
~[This
comment](https://github.com/astral-sh/uv/pull/6738/files#r1743510140)
remains unaddressed, however, the Child process does not have any other
methods besides kill() so I don't see how we can "preserve" the
interrupt call :/ I tried looking around but no luck.~ updated, this PR
is reduced to only handling SIGTERM propagation on unix machines, and
the sigterm call to the child is preserved by making use of the nix
package, instead of relying on tokio which only allowed for `kill()` on
a child process
## Test Plan
I tested this by building the docker container locally with these
changes and tagging it "myuv", and then using that as the base image in
uv-docker-example, (and ofc following the rest of the repro issues in
#6724. In my tests I see that ctrl-c in the docker-compose up command
exits the process almost immediately 👍
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
We're inconsistent with these -- sometimes it's `Error::Fetch` and
sometimes it's `Error::Download`. The message says download, so let's
just use that?
## Summary
This got moved to `InstallTarget`! Must've been an oversight not to
delete. I verified that no code was changed here since the date that we
moved it to `InstallTarget`.
## Summary
Just as we don't enforce tag compliance, we shouldn't enforce
`--no-build` when validating the lockfile. If we end up building from
source, the distribution database will correctly error.
Closes https://github.com/astral-sh/uv/issues/9016.
## Summary
At time of writing, `markupsafe==3.0.2` exists on the PyTorch index, but
there's
only a single wheel:
`MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`
Meanwhile, there are a large number of wheels on PyPI for the same
version. If the
user is on Python 3.12, and we return the incompatible PyTorch wheel
without
considering the PyPI wheels, PubGrub will mark 3.0.2 as an incompatible
version,
even though there are compatible wheels on PyPI.
Closes https://github.com/astral-sh/uv/issues/8922.
## Summary
We were making some incorrect assumptions in the extra-merging code for
universal `pip compile`. This PR corrects those assumptions and adds a
bunch of additional tests.
Closes https://github.com/astral-sh/uv/issues/8915.
## Summary
The basic issue here is that `uv add` will compute and store a hash for
each package. But if you later run `uv pip install` _after_ `uv cache
prune --ci`, we need to re-download the source distribution. After
re-downloading, we compare the hashes before and after. But `uv pip
install` doesn't compute any hashes by default. So the hashes "differ"
and we error.
Instead, we need to compute a superset of the already-existing and
newly-requested hashes when performing this re-download. (In practice,
this will always be SHA-256.)
Closes https://github.com/astral-sh/uv/issues/8929.
## Test Plan
```shell
export UV_CACHE_DIR="$PWD/cache"
rm -rf "$UV_CACHE_DIR" .venv .venv-2 pyproject.toml uv.lock
echo $(uv --version)
uv init --name uv-cache-issue
cargo run add --python 3.13 "pycairo"
uv cache prune --ci
rm -rf .venv .venv-2
uv venv --python python3.11 .venv-2
. .venv-2/bin/activate
cargo run pip install "pycairo"
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
auditwheel is capable of generating riscv64 wheels for manylinux_2_31
and above. Here we modify uv-platform-tags so that those wheels can be
installed using uv.
Fixes: #8889
## Test Plan
- ran `cargo nextest run` locally on an x86 machine
- also ran `cargo nextest run` locally on a riscv64 VM but there were a
fair few failures (with and without this patch)
- built a riscv64 uv wheel, installed it on a riscv64 VM and checked
that I could use the newly built version of uv to install
manylinux_2_35_riscv64 wheels.
## Summary
As an oversight, these arguments weren't being respected from the CLI or
elsewhere -- we always hit PyPI, ignored `--exclude-newer`, etc. It has
to do with the way that the `PipOptions` are setup -- there's a global
struct that we pass around everywhere and fill in with defaults, so
there's no type safety to guarantee that we provide whatever it is we
need to use in the command. The newer APIs are much better about this.
Closes#8927.
## Summary
In the example outlined in https://github.com/astral-sh/uv/issues/8884,
this removes an unnecessary `jupyter_contrib_nbextensions-0.7.0.tar.gz`
segment (replacing it with `src`), thereby saving 39 characters and
getting that build working on my Windows machine.
This should _not_ require a version bump because we already have logic
in place to "heal" partial cache entries that lack an unzipped
distribution.
Closes https://github.com/astral-sh/uv/issues/8884.
Closes https://github.com/astral-sh/uv/issues/7376.
## Summary
See: https://github.com/astral-sh/uv/issues/8884. We build in a
directory that's deep within the cache; to help with file name length
limits, we should build at the top-level of the cache.
We don't actually want users to see this, but we should be stripping it
anyway. Without this change, we show ranges in the debug logs that look
like `>=1.0.0, <1.0.0`, which is more confusing than helpful. (We may
want to post-process those debug ranges to remove these.)
After https://github.com/astral-sh/uv/pull/8797, we have spec-compliant
handling for local version identifiers and can completely remove all the
special-casing around it.
Implement a full working version of local version semantics. The (AFAIA)
major move towards this was implemented in #2430. This added support
such that the version specifier `torch==2.1.0+cpu` would install
`torch@2.1.0+cpu` and consider `torch@2.1.0+cpu` a valid way to satisfy
the requirement `torch==2.1.0` in further dependency resolution.
In this feature, we more fully support local version semantics. Namely,
we now allow `torch==2.1.0` to install `torch@2.1.0+cpu` regardless of
whether `torch@2.1.0` (no local tag) actually exists.
We do this by adding an internal-only `Max` value to local versions that
compare greater to all other local versions. Then we can translate
`torch==2.1.0` into bounds: greater than 2.1.0 with no local tag and
less than 2.1.0 with the `Max` local tag.
Depends on https://github.com/astral-sh/packse/pull/227.
Uses #6369 for test coverage.
Updates version file discovery to search up into parent directories.
Also refactors Python request determination to avoid duplicating the
user request / version file / workspace lookup logic in every command
(this supersedes the work started in
https://github.com/astral-sh/uv/pull/6372).
There is a bit of remaining work here, mostly around documentation.
There are some edge-cases where we don't use the refactored request
utility, like `uv build` — I'm not sure how I'm going to handle that yet
as it needs a separate root directory.
Not verifying the certificates of certain hosts should be supported for
all kinds of HTTPS connections, so we're making it a global option, just
like native tls. This fixes the remaining places using a client but were
not configuring allow insecure host.
Fixes#6983 (i think)
Closes#6983
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
These settings can only be defined in `pyproject.toml`, since they're
project-centric, and not _configuration_.
Closes https://github.com/astral-sh/uv/issues/8539.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: konsti <konstin@mailbox.org>
closes#6640
Could you suggest how I should test it?
(already tested locally)
---------
Co-authored-by: konstin <konstin@mailbox.org>
Co-authored-by: Charles Tapley Hoyt <cthoyt@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
e.g.
```
❯ echo "anyio" | cargo run -q -- pip compile - -v
DEBUG uv 0.4.30 (107ab3d71 2024-11-07)
DEBUG Starting Python discovery for a default Python
DEBUG Looking for exact match for request a default Python
DEBUG Searching for default Python interpreter in virtual environments, managed installations, or search path
DEBUG Found `cpython-3.12.7-macos-aarch64-none` at `/Users/zb/workspace/uv/.venv/bin/python3` (virtual environment)
```
```
❯ cargo run -q -- pip install anyio -v
DEBUG uv 0.4.30 (107ab3d71 2024-11-07)
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `cpython-3.12.7-macos-aarch64-none` at `/Users/zb/workspace/uv/.venv/bin/python3` (virtual environment)
```
vs
```
❯ uv pip install anyio -v
DEBUG uv 0.4.30 (61ed2a236 2024-11-04)
DEBUG Searching for default Python interpreter in system path
DEBUG Found `cpython-3.12.7-macos-aarch64-none` at `/Users/zb/workspace/uv/.venv/bin/python3` (virtual environment)
```
```
❯ echo "anyio" | uv pip compile - -v
DEBUG uv 0.4.30 (61ed2a236 2024-11-04)
DEBUG Starting Python discovery for a default Python
DEBUG Looking for exact match for request a default Python
DEBUG Searching for default Python interpreter in managed installations or system path
DEBUG Found `cpython-3.12.7-macos-aarch64-none` at `/Users/zb/workspace/uv/.venv/bin/python3` (virtual environment)
```
Very basic source distribution support. What's included:
- Include and exclude patterns (hard-coded): Currently, we have
globset+walkdir in one part and glob in the other. I'll migrate
everything to globset+walkset and some custom perf optimizations to
avoid traversing irrelevant directories on top. I'll also pick a glob
syntax (or subset), PEP 639 seems like a good candidate since it's
consistent with what we already have to support.
- Add the `PKG-INFO` file with metadata: Thanks to Code Metadata 2.2,
this metadata is reliable and can be read statically by external tools.
Example output:
```
$ tar -ztvf dist/dummy-0.1.0.tar.gz
-rw-r--r-- 0/0 154 1970-01-01 01:00 dummy-0.1.0/PKG-INFO
-rw-rw-r-- 0/0 509 1970-01-01 01:00 dummy-0.1.0/pyproject.toml
drwxrwxr-x 0/0 0 1970-01-01 01:00 dummy-0.1.0/src/dummy
drwxrwxr-x 0/0 0 1970-01-01 01:00 dummy-0.1.0/src/dummy/submodule
-rw-rw-r-- 0/0 30 1970-01-01 01:00 dummy-0.1.0/src/dummy/submodule/impl.py
-rw-rw-r-- 0/0 14 1970-01-01 01:00 dummy-0.1.0/src/dummy/submodule/__init__.py
-rw-rw-r-- 0/0 12 1970-01-01 01:00 dummy-0.1.0/src/dummy/__init__.py
```
No tests since the source distributions don't build valid wheels yet.
## Summary
Like pip, we now allow the semicolon to directly proceed the URL (but
require that it's either preceded or followed by a space):
```
# OK
./test.whl; sys_platform == 'darwin'
# OK
./test.whl ;sys_platform == 'darwin'
# Error
./test.whl;sys_platform == 'darwin'
```
Closes https://github.com/astral-sh/uv/issues/8831.
This PR simplifies the VersionSmall implementation a bit by utilizing
more constants. That is, if the bit-level format changes, *most* of
those changes should be implementable by just changing the constants.
Previously, you would need to audit and tweak the code as well. (The
exception here is `push_release`. If the release segment bit format is
changed, then that function will need to be tweaked. I didn't think it
was worth over-complicating things to make its implementation more
general.)
See https://github.com/astral-sh/uv/pull/8531#issuecomment-2442698889,
we hint users coming from twine to use `--check-url` instead.
> `uv publish` does not support `--skip-existing`, use `--check-url`
with the simple index URL instead.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Previously, we'd use the `--reinstall` flag to determine if we should
replace existing Python executables in the bin directory during an
install. There are a few problems with this:
- We replace executables we don't manage
- We can replace executables from other uv Python installations during
reinstall (surprising)
- We don't do the "right" thing when installing patch versions e.g.
installing `3.12.4` then `3.12.6` would fail without the reinstall flag
In `uv tool`, we have separate `--force` and `--reinstall` concepts.
Here we separate the flags (`--force` was previously just a
`--reinstall` alias) and add inspection of the existing executables to
inform a decision on replacement.
In brief, we will:
- Replace any executables with `--force`
- Replace executables for the same installation with `--reinstall`
- Replace executables for an older patch version by default
## Summary
This PR pulls in https://github.com/astral-sh/uv/pull/8263 and
https://github.com/astral-sh/uv/pull/8463, which were originally merged
into the v0.5 tracking branch but can now be committed separately, as
we've made `.env` loading opt-in.
In summary:
- `.env` loading is now opt-in (`--env-file .env`).
- `.env` remains supported on `uv run`, so it's meant for providing
environment variables to the run command, rather than to uv itself.
---------
Co-authored-by: Eduardo González Vaquero <47718648+edugzlez@users.noreply.github.com>
Running this test manually on the latest release shows that it still
emits a `python_version < '0'` marker:
$ uv-0.4.29 pip compile requirements.in -c constraints.txt
--annotation-style line --python 3.10 --universal | rg "< '0'"
Resolved 151 packages in 97ms
apache-airflow-providers-common-sql==1.19.0 ; python_version < '0' # via
apache-airflow-providers-sqlite
That is, even though this is a smaller test, it's still testing the same
bug.
This is about twice as fast on my machine. It's probably still worth
moving to a packse test, but this was quick to do.
This updates the surrounding code to use the new ResolverEnvironment
type. In some cases, this simplifies caller code by removing case
analysis. There *shouldn't* be any behavior changes here. Some test
snapshots were updated to account for some minor tweaks to error
messages.
I didn't split this up into separate commits because it would have been
too difficult/costly.
This type is intended to replace `ResolverMarkers`. The main difference
between them is that this type encapsulates more decision making by
un-exporting the different cases. So instead of callers needing to do
explicit case analysis depending on the type of resolver environment,
callers instead use methods that know how to do the right thing. In the
next commit, there are at least a few cases where this greatly
simplifies case analysis on the caller side.
The motivation for this type is to centralize decision making about
forking. In particular, we want to expand forking to include conflicting
groups instead of just `MarkerTree`. So to a certain extent, the
refactor here is about removing bare use of `MarkerTree` in favor of a
more purpose built type that encapsulates the forking logic.
The encapsulation is not quite perfect here. I expect to improve on it a
bit once we add support for conflicting groups.
This is split off from the subsequent commit (that makes use of
`ResolverEnvironment`) so that it's a bit easier to review the addition
in isolation.
`ExtraName` did implement `AsRef<str>`, but that should generally
only be used in a context with an `AsRef<str>` generic bound. If
you just want a `&str` from a concrete `ExtraName`, then a specific
method for that purpose should be used.
This PR is a revival of #3502, albeit in a much simpler form.
This would allow for different middlewares like authentication and such,
useful for if you want to deviate from the keychain authentication
methods when using uv as a library.
@zanieb I hope I made the changes as you noted you wanted to see them :)
Happy to add/change anything you need.
## Summary
At present, when we have a Python requirement and we see a wheel, we
verify that the Python requirement is compatible with the wheel. For
source distributions, though, we verify that both the Python requirement
_and_ the currently-installed version are compatible, because we assume
that we'll need to build the source distribution in order to get
metadata. However, we can often extract source distribution metadata
_without_ building (e.g., if there's a `pyproject.toml` with no dynamic
keys).
This PR thus modifies the source distribution handling to defer that
incompatibility ("We couldn't get metadata for this project, because it
has no static metadata and requires a higher Python version to run /
build") until we actually try to build the package. As a result, you can
now resolve source distribution-only packages using Python versions
below their `requires-python`, as long as they include static metadata.
Closes https://github.com/astral-sh/uv/issues/8767.
## Summary
* Env docs now support anchors, which allows sending a link to someone
with a direct reference to an env var or cross-reference them in the
docs.
* Marked additional env vars as hidden from the docs due to their
internal use
* Updates some tests still using literals to use the static env vars
## Test Plan
<img width="1370" alt="env_var_anchors"
src="https://github.com/user-attachments/assets/52ae1caa-5199-4798-9eb5-81b8f5b57c24">
## Summary
Resolves#8417
I've just begun learning procedural macros, so this PR is more of a
proof of concept. It's still a work in progress, and I welcome any
assistance or feedback.
## Summary
This PR improves the interaction of `--frozen` such that we reduce the
dependency on the `pyproject.toml` and increase the dependency on the
`uv.lock`. Specifically, we now read the list of workspace members from
the `uv.lock` rather than the `pyproject.toml`, which means we don't
need to discover the member `pyproject.toml` files in order to perform a
`uv sync --frozen --all-packages`.
## Summary
This PR enables `uv sync --all-packages` to sync all packages in a
workspace. It removes a common use-case for the legacy non-`[project]`
packages that we're trying to move away from.
Closes https://github.com/astral-sh/uv/issues/8724.
This PR fixes a bug where it was possible for dependencies to be
included in a final resolution with markers that always evaluate to
false. Specifically, `python_version < '0'`.
While we do filter based on Python markers during forking, it turns out
that the markers for each fork are "combined" *after* this filtering
step. But the process of combination can result in a more specific
marker that is always false for the configured Python requirement. This
could result in dependencies with markers that are always false (like
`python_version < '0'`) appearing in the resolution.
The first commit in this PR adds a regression test (with an undesirable
result), and the second commit fixes the regression and updates the
test.
Fixes#8676
This still utilizes the RFC 2822 datetime formatter, but utilizes new
methods [added in jiff 0.1.14] to emit timestamps in a format strictly
compatible with RFC 9110.
It seems like most HTTP servers were pretty flexible and supported RFC
2822 datetime formats, but #8747 shows at least one case where that
isn't true. Given that the [MDN docs prescribe RFC 9110], we defer to
them.
Fixes#8747
[added in jiff 0.1.14]: https://github.com/BurntSushi/jiff/pull/154
[MDN docs prescribe RFC 9110]:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
## Summary
It turns out that when locking, we were only taking the groups from the
root `pyproject.toml` into account, and ignoring groups that were only
defined in a workspace member.
## Summary
By default, `uv tree` shows the full workspace, not _just_ the root. If
the root depended on a workspace member as a dev dependency, then we'd
still show it as `(group: dev)` in `uv tree` even if you passed
`--no-dev`, because we weren't filtering the edges in the right place.
This is still somewhat confusing, because if `root` depends on workspace
member `child` as a dev dependency, `uv tree --no-dev` still shows both.
Closes https://github.com/astral-sh/uv/issues/8719.
Incorporating #8637 into #8458
- Adds `python-managed` feature selection to Windows CI for `python
install` tests
- Adds trampoline sniffing utilities to `uv-trampoline-builder`
- Uses a trampoline to install Python executables into the `PATH` on
Windows
Pulling out of https://github.com/astral-sh/uv/pull/8650 for
readability.
Trying to clean this up to simplify extensions in the future. This is
not a strict refactor, there are behavioral changes here.
- Adds some structs for managing state.
- Addresses some likely inconsistent behavior for weird edge-cases.
- We fill platform information before checking if a request is
satisfied.
- We error earlier if we can't find a download for the request, i.e.,
even if you somehow have it installed.
- Only reports versions as uninstalled if a download actually replaces
them.
- Moves some of the default output to tracing messages.
- Even if an installation was already satisfied, we'll check that it is
setup properly
When resolving workspace dependencies (from one workspace member to
another) from a workspace that's in git, we need to emit these
transitive dependencies as git dependencies, not path dependencies as
all other workspace deps. This fixes a bug where we would treat them as
path dependencies inside the checkout directory, leading either to
clashes (between a local path and another direct git dependency) or
invalid lockfiles (referencing the checkout dir in the lockfile when we
should be referencing the git repo).
Fixes#8087Fixes#4920Fixes#3936 since we needed that information anyway
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Updates `uv python install` to link `python3.x` in the executable
directory (i.e., `~/.local/bin`) to the the managed interpreter path.
Includes
- #8569
- #8571
Remaining work
- #8663
- #8650
- Add an opt-out setting and flag
- Update documentation
## Summary
These use really heavy test packages, like SciPy, NumPy, scikit-learn --
and packages with large dependency trees, like packse.
I removed a few redundant tests, and replaced the tests with smaller
packages.
Closes https://github.com/astral-sh/uv/issues/8674.
Previously, when doing a `uv pip` resolution, we would only return the
first entry in the map. But there should only ever be one entry, or else
we would have incompatible dependencies. So we can collapse the case
with the "one universal fork" case.
(I found this while doing some refactoring of how we handle forking, and
collapsing these cases simplifies some of that refactoring work.)
Previously, when tests were run in `~/.local/share/uv`, the behavior of
some tests could be impacted by a git repository in `~` (as on my
system). To avoid this, we set `GIT_CEILING_DIRECTORIES` to forcefully
prevent git from climbing out of its test directory to look for parent
git repositories.
When this code was written, we didn't have "proper" disjointness checks,
and so simple equality was used instead. Arguably disjointness checks
are
more correct, and this would also simplify some case analysis in an
ongoing
refactor.
Currently, our trampoline is used to convert `<command> [args]` to
`python <command> [args]` for script entrypoints installed into virtual
environments. For #8458, it'd be nice to convert a shim `python3.12
[args]` to `python [args]`. Here, we modify the trampolines to support
this use-case.
The only change we really need here is to avoid injecting `<command>`
into the child process. We change the "magic number" at the end of the
trampoline executables from `UVUV` to `UVSC` and `UVPY` which define
"script" and "python" variants to the trampoline. We then omit the
`<command>` injection in the latter case. We also omit writing the zip
script payload.
To support construction of the new variant, a new
`uv-trampoline-builder` crate is introduced — this avoids requirements
on `uv-install-wheel` in future work. I also use `uv-trampoline-builder`
to consolidate some of the test setup for `uv-trampoline`.
There should be no backwards compatibility concerns, since trampolines
are fully self-referential.
I rebased to fix the commits at the end, as this took many iterations to
get working via CI. This should roughly be reviewable by commit if you
prefer.
## Summary
Unfortunately, it looks like we lost
https://github.com/astral-sh/uv/pull/8501 somewhere in a bad rebase.
This PR re-adds the change, with compatibility for those lockfiles
created in v0.4.27. I'm not certain we should actually merge this. It
might be less painful and confusing to just bite the bullet on the
change.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Refs:
- #8626
## Summary
Current documentation incorrectly suggests that the macOS cache
directory location is `$HOME/Library/Caches/uv`, but that changed in:
- #5806
Updates docs to say this instead:
> <p>Defaults to <code>$HOME/.cache/uv</code> on macOS,
<code>$XDG_CACHE_HOME/uv</code> or <code>$HOME/.cache/uv</code> on
Linux, and <code>%LOCALAPPDATA%\uv\cache</code> on Windows. The <code>uv
cache dir</code> command will show the location of the cache
directory.</p>
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
The changes in this commit introduce the `UV_NO_PROGRESS` environment
variable as an alternative way to control progress output suppression in
uv-cli, equivalent to using the `--no-progress` flag. This enhancement
simplifies configuration in CI environments and automated scripts by
eliminating the need to detect whether the script is running in a CI
environment.
Previously, disabling progress output required either passing the
`--no-progress` flag directly or implementing script logic to detect CI
environments and conditionally add the flag. With this change, users can
now simply set `UV_NO_PROGRESS=true` in their environment to achieve the
same effect.
The changes include:
- Adding the `UV_NO_PROGRESS` environment variable to the `EnvVars`
struct in `crates/uv-static/src/env_vars.rs`.
- Updating the `GlobalArgs` struct in `crates/uv-cli/src/lib.rs` to
include a new `no_progress` field that is bound to the `UV_NO_PROGRESS`
environment variable.
- Adding documentation for the new `UV_NO_PROGRESS` environment variable
in `docs/configuration/environment.md`.
## Test Plan
After creating a uv project using `uv init` in a temp directory in this
project:
```
cargo run cache clean && cargo run venv && UV_NO_PROGRESS=false cargo run sync
cargo run cache clean && cargo run venv && cargo run sync
```
produce the expected default behavior
```
cargo run cache clean && cargo run venv && UV_NO_PROGRESS=false cargo run sync
```
produces the same behavior as having the `--no-progress` flag.
## Summary
This PR removes an unnecessary `return` statement from Rust code in the
Maturin template. This makes the generated code more idiomatic and
prevents a Clippy warning.
## Test Plan
This change was tested both in unit tests and by manually creating a
Maturin project.
The shared arguments were resetting the `UV_PYTHON_INSTALL_DIR`,
breaking the intent of the test.
Added some uninstall tests too, needed later for #8571
## Summary
It turns out we were omitting empty dependency groups from the lockfile
metadata, which was then causing us to reject locks when empty groups
were defined.
We now include them (that section of the lock is meant to be a true
representation of the metadata, and an empty-but-defined group is
different from an absent group), though we can ignore them for
validation, since it doesn't affect any behavior.
Closes https://github.com/astral-sh/uv/issues/8581.
## Summary
We already support `tool.uv.dev-dependencies` in the legacy
non-`[project]` projects. This adds equivalent support for
`[dependency-groups]`, e.g.:
```toml
[tool.uv.workspace]
[dependency-groups]
lint = ["ruff"]
```
## Summary
`uv add --dev` now updates the `dependency-groups.dev` section, rather
than `tool.uv.dev-dependencies` -- unless the dependency is already
present in `tool.uv.dev-dependencies`.
`uv remove --dev` now removes from both `dependency-groups.dev` and
`tool.uv.dev-dependencies`.
`--dev` and `--group dev` are now treated equivalently in `uv add` and
`uv remove`.
This PR adds support for `tool.uv.default-groups`, which defaults to
`["dev"]` for backwards-compatibility. These represent the groups we
sync by default.
## Summary
We have to iterate over all user-defined dependencies here. We were
missing the new `[dependency-groups]` section.
Part of https://github.com/astral-sh/uv/pull/8272.
Part of #8090
Unblocks https://github.com/astral-sh/uv/pull/8274
Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
Part of #8090
Adds the ability to read group inclusions (`include-group = <name>`) in
the `pyproject.toml`. Resolves groups into concrete dependencies for
resolution.
See https://github.com/astral-sh/uv/pull/8110 for a bit more commentary
on deferred work.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Part of #8090
Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.
There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.
Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
Part of #8090
Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.
Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
## Summary
These two sentences in the docs for `--publish-url` seem to basically be
duplicates:
3eda248ef5/crates/uv-cli/src/lib.rs (L4616-L4618)
I found the first to be easier to read, so this commit removes the
second.
## Test Plan
No tests, change is docs-only.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
For example, in:
```toml
[tool.uv]
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
We can just omit `[tool.uv]`.
## Summary
We were including dependencies that were only included by a dependency
that isn't relevant on the current platform (i.e., we were enforcing the
"current environment" at one level, but not transitively).
Closes https://github.com/astral-sh/uv/issues/8516.
## Summary
Historically, we haven't enforced schema versions. This PR adds a
versioning policy such that, if a uv version writes schema v2, then...
- It will always reject lockfiles with schema v3 or later.
- It _may_ reject lockfiles with schema v1, but can also choose to read
them, if possible.
(For example, the change we proposed to rename `dev-dependencies` to
`dependency-groups` would've been backwards-compatible: newer versions
of uv could still read lockfiles that used the `dev-dependencies` field
name, but older versions should reject lockfiles that use the
`dependency-groups` field name.)
Closes https://github.com/astral-sh/uv/issues/8465.
With a change like https://github.com/astral-sh/uv/pull/8458, we really
need tests for these.
I'm just going to take the possible performance hit of these slow tests
and deal with optimizing them separately.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Fix the flag description: `to detect and with` --> `to detect packages
with`
This PR adds support for `uv lock --dry-run`, as described in issue
#6408.
One thing to note: this functionality, as implemented, isn't limited to
`-U` (if someone adds a dependency to the project's `pyproject.toml`,
the plan will include these changes).
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Previously, `uv tree --package` had some strange behavior due to how we
were computing the root nodes. This PR refactors the entire
implementation to use `petgraph` so we can do proper operations on a
graph structure.
Closes https://github.com/astral-sh/uv/issues/8382.
## Summary
This PR delays the removal of an existing version after downloading the
new version when running `uv python install --reinstall`.
If the download fails, we can keep the existing version working.
## Test Plan
```console
$ cargo run -- python install 3.13
$ cargo run -- python install --reinstall 3.13 # when downloading, `ctrl-c` to interrupt
$ cargo run -- python list
```
## Summary
Instead of creating a new entry, we should reuse the existing entry (to
preserve decor); similarly, we should avoid overwriting fields that are
already "correct".
Closes https://github.com/astral-sh/uv/issues/8483.
## Summary
We no longer need this struct; we bumped the cache bucket version
anyway, so the `Timestamp` variant is never encountered. This means we
get real Serde error messages.
Closes https://github.com/astral-sh/uv/issues/8488.
## Summary
This is part of making
https://github.com/astral-sh/uv/issues/7299#issuecomment-2385286341
better. You can now use `tool.uv.dependency-metadata` for direct URL
requirements. Unfortunately, you _must_ include a version, since we need
one to perform resolution.
## Summary
Look for a system level uv.toml config file under `/etc/uv/uv.toml` or
`C:\ProgramData`.
This PR is to address #6742 and start a conversation.
## Test Plan
This was tested locally manually on MacOS. I am happy to contribute
tests once we settle on the approach.
cc @thatch
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
If the user has an upper-bound in a `requires-python`, we don't
correctly narrow it during resolution. We should be narrowing based on
the intersection.
Closes#8297.
## Summary
The desired behavior for `uv tree` and `uv pip list` with `-q | --quiet`
flag is
https://github.com/astral-sh/uv/issues/8379#issuecomment-2425093709 to
still produce output. This is implemented here.
Closes https://github.com/astral-sh/uv/issues/8379
## Test Plan
Use `uv tree -q` or `uv pip list -q` on any uv project setup and expect
the corresponding output.
Added tests for that as well.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Now, we use four space (rather than one space) for cases like:
```toml
dependencies = [ # comment 0
# comment 1
"anyio==3.7.0", # comment 2
# comment 3
]
```
## Summary
Rather than relying on the distribution and package URL being the same
(which isn't true for Git dependencies), we can just use the
intersection of the markers directly.
Closes https://github.com/astral-sh/uv/issues/8381.
Cherry-picked from #8347
Might fix https://github.com/astral-sh/uv/issues/6940 — I'm not seeing a
failure over there after this change. I think there may be some problem
with concurrent reads of junctioned files on the DevDrive? It's really
hard to say.
We might lose some important test coverage with this change. I'm not
sure what to do about that either.
Cherry-picked from https://github.com/astral-sh/uv/pull/8347
Seems generally really helpful to see the unfiltered snapshot when a
test fails. Especially when debugging filters on Windows.
## Summary
This PR is is to address the problem when the same-line comments in
`pyproject.toml` could be found in unpredictable positions after `uv
add` or `remove` reformats the `pyproject.toml` file.
Introduced the `Comment` structure in `pyproject_mut` module to
distinguish "same-line" comments and "full-line" comments while
reformatting, because logic for them differs.
Sorry, the implementation could be clumsy, I'm just learning Rust, but
it seems to work 😅
Closes https://github.com/astral-sh/uv/issues/8343
## Test Plan
Added the new test:
`add_preserves_comments_indentation_and_sameline_comments`
To test followed the actions from the issue ticket
https://github.com/astral-sh/uv/issues/8343
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Going forward, we're going to provide better versioning guarantees
around using the same cache across multiple uv versions, so this PR
updates the docs to reflect that. It also bumps the `sdists-` version to
fix the inconvenience demonstrated in
https://github.com/astral-sh/uv/issues/8367.
Closes https://github.com/astral-sh/uv/issues/8367.
## Summary
Resolves#7685
## Test Plan
```console
$ echo "this is an invalid netrc" > .netrc
$ NETRC=.netrc cargo run -- pip install anyio --index-url https://pypi-proxy.fly.dev/basic-auth/simple --strict -v
DEBUG uv 0.4.24 (f4d5fba61 2024-10-19)
DEBUG Searching for default Python interpreter in system path or `py` launcher
DEBUG Found `cpython-3.11.2-windows-x86_64-none` at `D:\Projects\Rust\uv\.venv\Scripts\python.exe` (virtual environment)
DEBUG Using Python 3.11.2 environment at .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: anyio
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.2
DEBUG Solving with target Python version: >=3.11.2
DEBUG Adding direct dependency: anyio*
DEBUG No cache entry for: https://pypi-proxy.fly.dev/basic-auth/simple/anyio/
WARN Error reading netrc file: parsing error: bad toplevel token 'this' (line 1) in the file '.netrc'
DEBUG Searching for a compatible version of anyio (*)
DEBUG No compatible version found for: anyio
× No solution found when resolving dependencies:
╰─▶ Because anyio was not found in the package registry and you require anyio, we can conclude that your
requirements are unsatisfiable.
hint: An index URL (https://pypi-proxy.fly.dev/basic-auth/simple) could not be queried due to a lack of valid
authentication credentials (401 Unauthorized).
DEBUG Released lock at `D:\Projects\Rust\uv\.venv\.lock`
error: process didn't exit successfully: `target\debug\uv.exe pip install anyio --index-url https://pypi-proxy.fly.dev/basic-auth/simple --strict -v` (exit code: 1)
```
Part of https://github.com/astral-sh/uv/issues/2777
I noticed we're seeing "Python ABI" _a lot_ in error messages which I
did not expect. This improves a common case by being a little more
specific.
Basically, if username-only authentication came from the _cache_ instead
of being present on the _request URL_ to start, we'd end up ignoring it
during password lookups which breaks keyring.
Includes some cosmetic changes to the logging and commentary in the
middleware, because I was confused when reading the code and logs.
The docs reference `UV_INDEX_`, but the code actually uses
UV_HTTP_BASIC_ as the prefix for environment variable credentials.
See PR #7741
Code is at
https://github.com/astral-sh/uv/blob/main/crates/uv-static/src/env_vars.rs#L163
```rust
/// Generates the environment variable key for the HTTP Basic authentication username.
pub fn http_basic_username(name: &str) -> String {
format!("UV_HTTP_BASIC_{name}_USERNAME")
}
/// Generates the environment variable key for the HTTP Basic authentication password.
pub fn http_basic_password(name: &str) -> String {
format!("UV_HTTP_BASIC_{name}_PASSWORD")
}
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
The hack in pip itself only modifies entry points called
`pip<number>.<number>` and `easy_install-<number>.<number>`, uv
previously dropped too many items including any of the form
`foo.<number>`.
Found while trying to install `memray` which somewhat notably does not
provide an abi3 wheel, so the installed, suffixed script matches. At a
minimum, this makes the installed files match the `entry_points.txt`
more than it did previously, which makes `pickley` happy.
## Test Plan
New test provided for previously-untested code.
Closes https://github.com/astral-sh/uv/issues/8228
e.g., on this branch
```
❯ uv python install 3.13t 3.13
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13 --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13t --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
and on main
```
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
Installed 3 packages in 12ms
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
I want to add more test coverage around this, but I've noticed the
free-threaded discovery tests are a bit off as-is and it'll be a bigger
task. I think the recent bugs around discovery indicate we should invest
more into that test framework.
Extends #7959
While I was looking at that message, I noticed I didn't love the
readability of the existing message and opted to follow-up with a change
to them both.
When patch version isn't specified and a matching version is referenced,
it will default patch to 0 which could be unclear/confusing. This PR
warns the user of that default.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
The first part of this issue
https://github.com/astral-sh/uv/issues/7426. Will tackle the second part
mentioned (`~=`) in a separate PR once I know this is the correct way to
warn users.
## Test Plan
<!-- How was it tested? -->
Unit tests were added
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
We shouldn't show these in `uv add`, especially when the thing we're
adding is about to have a lower-bound put on it. Now, we only show these
when the user runs `uv lock` or `uv sync`.
## Summary
If you pass a named index via the CLI, you can now reference it as a
named source. This required some surprisingly large refactors, since we
now need to be able to track whether a given index was provided on the
CLI vs. elsewhere (since, e.g., we don't want users to be able to
reference named indexes defined in global configuration).
Closes https://github.com/astral-sh/uv/issues/7899.
## Summary
This PR adds an index pin with `uv add` when the user provides exactly
one named index. We don't pin if the user provides an unnamed index, or
if they provide multiple indexes.
We probably _could_ pin on multiple indexes by writing the sources
_after_ resolution, if that's desirable. But we have no idea which index
the user _expects_ each package to come from.
Possible extensions:
- `uv add --no-pin` to avoid this pinning.
- Warn if they provide a single, unnamed index? I'm not sure if that's
worth a warn. Open to input.
## Summary
This was already properly handled, but the operation itself was in a
`debug_assert!`, so it wasn't running at all in production builds...
Closes https://github.com/astral-sh/uv/issues/8208.
## Summary
This PR lifts the restriction that a package must come from a single
index. For example, you can now do:
```toml
[project]
name = "project"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["jinja2"]
[tool.uv.sources]
jinja2 = [
{ index = "torch-cu118", marker = "sys_platform == 'darwin'"},
{ index = "torch-cu124", marker = "sys_platform != 'darwin'"},
]
[[tool.uv.index]]
name = "torch-cu118"
url = "https://download.pytorch.org/whl/cu118"
[[tool.uv.index]]
name = "torch-cu124"
url = "https://download.pytorch.org/whl/cu124"
```
The construction is very similar to the way we handle URLs today: you
can have multiple URLs for a given package, but they must appear in
disjoint forks. So most of the code is just adding that abstraction to
the resolver, following our handling of URLs.
Closes#7761.
## Summary
The behavior is as follows:
- If you provide `--index` or `--default-index` on the command-line, we
add those indexes to the `pyproject.toml` (with names, if provided, as
in `--index pytorch=https://download.pytorch.org/whl/cu121`.
- If you provide `--index-url` or `--default-index`, we warn, but don't
add the indexes to the file. (This seems wrong -- why not add them?)
- If you provide an index with a name or URL that already exists, we
remove that entry, and add the new index to the top of the list (since
it now has highest priority).
- If you provide a `--default-index`, and an index already has `default
= true`, we remove that entry, since it won't be used anymore.
We do _not_ pin packages to specific indexes yet.
## Summary
This PR enables users to provide index credentials via named environment
variables.
For example, given an index named `internal` that requires a username
(`public`) and password
(`koala`), you can define the index (without credentials) in your
`pyproject.toml`:
```toml
[[tool.uv.index]]
name = "internal"
url = "https://pypi-proxy.corp.dev/simple"
```
Then set the `UV_INDEX_INTERNAL_USERNAME` and
`UV_INDEX_INTERNAL_PASSWORD`
environment variables, where `INTERNAL` is the uppercase version of the
index name:
```sh
export UV_INDEX_INTERNAL_USERNAME=public
export UV_INDEX_INTERNAL_PASSWORD=koala
```
## Summary
This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.
Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.
### Index priority
Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.
Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.
You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.
### Index pinning
If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[tool.uv.sources]
torch = { index = "pytorch" }
```
If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch" }
```
Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.
(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)
### Default index
By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```
### Name reuse
If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.
For example, given:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```
The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.
Closes#171.
## Future work
- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
{ index = "cpu", markers = "sys_platform == 'darwin'" },
{ index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
## Summary
Cache the path to git executable in a `LazyLock` and reuse it throughout
the process. This might reduce some costs on finding the git executable.
When building a source distribution to a wheels, we perform the build
inside a temporary directory inside the output directory. By default,
the output directory is `dist/` in the repository root. This temp dir
placement allows us to move the final wheel to the output directory
instead of copying it (a temp dir might be on another device, which
means we need to copy instead of moving).
Some build backends such as hatchling traverse upwards from the current
directory (the source dist build location) looking for gitignore files
to consider. By adding a gitignore in `dist/` with `*`, we caused
hatchling to ignore all files in our temporary build directory below it,
causing empty wheels. To prevent this, we add a `.git` file as a phony
git root. We are already using this trick successfully in the cache.
Hatchling sees this `.git` file, considers it a boundary and does not
traverse up to `dist/.gitignore`.
Fixes#8200
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Closes https://github.com/astral-sh/uv/issues/8213
I didn't mean to remove these when updating the regular expression.
Arguably, they shouldn't be used anymore, but we should make that choice
with intention.
As mentioned in https://github.com/astral-sh/uv/issues/8189
We only checked if an interpreter was free-threaded _when_ free-threaded
variants were requested. But we should not use free-threaded
interpreters unless explicitly requested.
## Summary
This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.
I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.
This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.
I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.
I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.
Did I miss any? Are their initial docs cohesive?
Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.
## Test Plan
Existing tests
## Summary
If you `uv add` a Git dependency, we resolve it twice:
While we need to avoid sharing state between `lock` and `sync` (see the
large TODO that moved in this change), we should prioritize sharing
state between different resolver operations.
Allow '*' as a value to match all hosts, and provide
`reqwest_blocking_get` for uv tests, so that they also respect
UV_INSECURE_HOST (since they respect `ALL_PROXY`).
This lets those tests pass with a forward proxy - we can think about
setting a root certificate later so that we don't need to disable
certificate verification at all.
---
I tested this locally by running:
```bash
GIT_SSL_NO_VERIFY=true ALL_PROXY=localhost:8080 UV_INSECURE_HOST="*" cargo nextest run sync_wheel_path_source_error
```
With my forward proxy showing:
```
2024-10-09T18:20:16.300188Z INFO fopro: Proxied GET cc2fedbd88a6546c1727ae13fa977a/cffi-1.17.1-cp310-cp310-macosx_11_0_arm64.whl (headers 480.024958ms + body 92.345666ms)
2024-10-09T18:20:16.913298Z INFO fopro: Proxied GET https://pypi.org/simple/pycparser/ (headers 509.664834ms + body 269.291µs)
2024-10-09T18:20:17.383975Z INFO fopro: Proxied GET 5f610ebe4298517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl.metadata (headers 443.184208ms + body 2.094792ms)
```
## Summary
When trying out standalone scripts I noticed a warning that said
`--no_readme` is a no-op when I provided `--no-readme`.
I searched for this "--\w+_" pattern in the codebase and found a similar
typo in warnings in other places, so I fixed them here.
## Test Plan
no plan, since these commands are mainly for interactive use I would
assume nobody parses out warnings about unnecessary options?
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html
Before that, there were 91 separate integration tests binary.
(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
## Summary
Related issues: #8009#7549
Although `PYTHONIOENCODING=utf-8` forces python to use UTF-8 for
`stdout`/`stderr`, it can't prevent code like
`sys.stdout.buffer.write()` or `subprocess.call(["cl.exe", ...])` to
bypass the encoder. This PR uses lossy UTF-8 conversion to avoid
decoding error.
## Alternative
Using `bstr` crate might be better since it can preserve original
information. Or we should follow the Windows convention, unset
`PYTHONIOENCODING` and decode with system default encoding.
## Test Plan
Running locally with non-ASCII character in `UV_CACHE_DIR` works fine,
but I have no unit test plan. Testing locale problem is hard :(
This is to address my own issue #7908
## Summary
This change makes use of the `clap` value_delimiter parser to populate
the `with` `Vec<String>` which currently can either only be empty or
with 1 value for each `--with` flag.
This makes use of the current code structure but allows for multiple
arguments with a single `--with` flag.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Can be tested with the following CLI:
```bash
target/debug/uv tool run --with numpy,polars,matplotlib ipython -c "import numpy;import polars;import matplotlib;"
```
And former behavior of multiple `--with` flags are kept
```bash
target/debug/uv tool run --with numpy --with polars --with matplotlib ipython -c "import numpy;import polars;import matplotlib;"
```
<!-- How was it tested? -->
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Fixes#8097. One challenge is that the `Pep723Script` is used for both
reading
and writing the metadata, so I wasn't sure about how to handle
`script.write`
when stdin (currently just ignoring it, but maybe we should raise an
error?).
## Test Plan
Added a test case copying the `test_stdin` with PEP 723 metadata.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
First off, congratulations on the 0.3 release! The PEP 723 standalone
scripts support is awesome, and I can already imagine a long tail of
little scripts of my own that would benefit from this functionality.
## Background
I really like the Deno CLI's support for running and installing remote
scripts.
```
deno run <url>
```
```
deno install --name foo <url>
```
I can see parallels with `uv run` and `uvx`. After mentioning this on
Discord, @zanieb suggested I could take a stab at a PR to implement
similar functionality for uv.
## Summary
This PR attempts to add support for executing remote standalone scripts
directly with `uv run`. While this is already possible by downloading
the script (i.e., via curl/wget) and then using uv run, having direct
support would be convenient.
The proposed functionality is:
```sh
uv run <url>
```
Another addition/alternative could be to support running scripts via
stdin:
```sh
curl -sL <url> | uv run -
```
But that is not implemented in this PR.
## Test Plan
I noticed that GitHub and `files.pythonhosted.org` URLs are used in some
of the tests. I've created a personal [GitHub
Gist](https://gist.github.com/manzt/cb24f3066c32983672025b04b9f98d1f)
with the example from PEP 723 for now to test this functionality.
~However, I couldn't figure out how to get the `with_snapshot` config
filter to filter out the tempfile path, so the test is currently
failing. Any assistance with this would be appreciated.~
## Notes
I'm not totally pleased with the implementation of this PR. I think it
would be better to handle the case earlier (and probably reuse the
cache), and avoid mutation, but since run command requires a local path
this was the simplest implementation I could come up with.
I know that performance is paramount with uv so I totally understand if
this requires a different approach or something more explicit to avoid
"inferring" the path. I'm just taking this as an opportunity to learn a
little more Rust and acquaint myself with the code base. cheers!
---------
Co-authored-by: Andrew Gallant <jamslam@gmail.com>
## Summary
In the routine we use to verify whether the lockfile is up-to-date, we
sometimes have to resolve package metadata. If that resolution step
fails, the resolver is left in a bad state, as various tasks are marked
as pending despite the error. Treating that as a recoverable failure
thus leads to a deadlock.
This PR modifies the errors to be treated as fatal.
I think a more holistic fix here would be to add some kind of guard to
ensure that any tasks that fail are no longer marked as pending (or
enforce this in the type system).
Closes https://github.com/astral-sh/uv/issues/8074.
## Summary
We can't rely on reading these from the `pyproject.toml`; instead, we
resolve the project metadata (which will typically just require reading
the `pyproject.toml`, but will go through our standard metadata paths).
Closes https://github.com/astral-sh/uv/issues/8071.
This commit fixes a bug where disjointness checking didn't always
satisfy commutativity. And it *should*. The `is_disjoint_commutative`
test added here demonstrates a regression test. Before this commit,
its second assertion failed.
That is, given `m1 = extra == "A" and extra != "B"` and
`m2 = extra == "A"`, we were saying that m1 was disjoint with
m2 (wrong) but that m2 was not disjoint with m1 (right).
It turned out that this was a "simple" matter of not using the
correct parent node when calling negation. Likely just a
transcription snafu.
This bug does not seem restricted in scope to extras, which is
how I found it, so it's not clear why we haven't noticed it until
now. I noticed it because I was formulating markers in a similar
format for resolver forking based on conflicting extras, and this
resulted in incorrectly filtering out dependencies due to `is_disjoint`
returning a false positive.
I found myself using this more verbose representation to double
check that there wasn't any other "hidden" state occurring in
markers, and that the graph debug display wasn't hiding anything
that I was missing.
## Summary
Now that `uv-install-wheel` output shows up in `--verbose`, lets leave
`debug!` to logs that users might want to see. Logging _every_ file we
install seems excessive.
Fixes: #8058
## Test Plan
Integration test (but only for Unix, because symlinks on Windows require
admin privs. Plus, they are not really all that idiomatic on Windows)
## Summary
These values can include spaces when passed on the command-line... Clap
doesn't give us a way to provide a value separator for _only_ an
environment variable (as is pip's behavior), so I think we're stuck
using comma-separated for here right now.
See, e.g., https://github.com/clap-rs/clap/discussions/3796.
Closes#8057.
## Summary
The issue here is that, if you user has a `requires-python` like `>=
3.7, != 3.8.5`, this gets expanded to the following bounds:
- `[3.7, 3.8.5)`
- `(3.8.5, ...`
We then convert this to the specific `>= 3.7, < 3.8.5, > 3.8.5`. But the
commas in that expression are conjunctions... So it's impossible to
satisfy? No version is both `< 3.8.5` and `> 3.8.5`.
Instead, we now preserve the input `requires-python` and just
concatenate the terms, only using PubGrub to compute the _bounds_.
Closes https://github.com/astral-sh/uv/issues/7862.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
- Adds detail to the `uv tool run --help` CLI that lets users know about
the shorthand, `uvx`
## Test Plan
<!-- How was it tested? -->
Building and running CLI
```
…📝✓] via 🐋 orbstack via 🎁 v0.4.16 via pyenv via ⚙️ v1.81.0on ☁️ (us-east-2) took 3s
➜ ./target/debug/uv tool run --help
Run a command provided by a Python package. Also available via the alias `uvx`.
Usage: uv tool run [OPTIONS] [COMMAND]
...
You can also use `uvx` as an alias for `uv tool run`.
Use `uv help tool run` for more details.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Closes#7977. Makes `PythonDownloadRequest` account for the prerelease
part if allowed. Also stores the prerelease in `PythonInstallationKey`
directly as a `Prerelease` rather than a string.
## Test Plan
Correctly picks the relevant prerelease (rather than picking the most
recent one):
```
λ cargo run python install 3.13.0rc2
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
Running `target/debug/uv python install 3.13.0rc2`
Searching for Python versions matching: Python 3.13rc2
cpython-3.13.0rc2-macos-aarch64-none ------------------------------ 457.81 KiB/14.73 MiB ^C
λ cargo run python install 3.13.0rc3
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
Running `target/debug/uv python install 3.13.0rc3`
Searching for Python versions matching: Python 3.13rc3
Found existing installation for Python 3.13rc3: cpython-3.13.0rc3-macos-aarch64-none
```
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
## Summary
This PR adds the ability to list available scripts in the environment
when `uv run` is invoked without any arguments.
It somewhat mimics the behavior of `rye run` command
(See https://rye.astral.sh/guide/commands/run).
This is an attempt to fix#4024.
## Test Plan
I added test cases. The CI pipeline should pass.
### Manuel Tests
```shell
❯ uv run
Provide a command or script to invoke with `uv run <command>` or `uv run script.py`.
The following scripts are available:
normalizer
python
python3
python3.12
See `uv run --help` for more information.
```
---------
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Improve hints when using the simple index URL instead of the upload URL
in `uv publish`. This is the most common confusion when publishing, so
we give it some extra care and put it more centrally in the CLI help.
Fixes#7860
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
https://github.com/astral-sh/uv/pull/7766 banned using `uv add` to
create self-dependencies in the `dev` section which breaks `uv add --dev
.[extra]` which is a fair use-case for adding a self-dependency.
Maybe we should only allow this if the added requirement includes an
extra group? Otherwise it's a bit weird.
## Summary
Closes#7841. If there are other env vars that would also benefit from
this value parser, please let me know and I can add them to this PR.
## Test Plan
When running the same example from the linked issue, it now works:
```
UV_PYTHON= cargo run -- init x
Compiling ...
Finished `dev` profile [unoptimized + debuginfo] target(s) in 29.06s
Running `target/debug/uv init x`
Initialized project `x` at `/Users/krishnanchandra/Projects/uv/x`
```
## Summary
This PR adds support for the `UV_FIND_LINKS` environment variable as an
alternative to the `--find-links` command-line option, as requested in
#1839.
## Test Plan
A unit test was added to validate that setting `UV_FIND_LINKS` provided
the same result as a link provided with the `--find-links` command-line
option.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Unlike `cp36-...`, which requires exactly CPython 3.6, `py36-none` is
compatible with all versions starting at Python 3.6.
Note that `py3x-none` should not be used. Instead, use `py3-none` with
`requires-python`.
Fixes#7800
## Summary
PythonDownloadKey (cpython-3.13.0rc3-darwin-aarch64-none) and
PlatformTriple in `fetch-download-metadata.py` have a slight
inconsistency in the ordering of `os` and `arch`. In PythonDownloadKey,
`os` precedes `arch`, while in PlatformTriple, `arch` comes before
`platform` (equivalent to os). This difference in ordering affects the
sorting logic, giving arch higher priority than platform in the
`download-metadata.json` file, leading to a little bit of unexpected
order of entries.
Before:
<img width="676" alt="image"
src="https://github.com/user-attachments/assets/adb24a2e-da70-4a09-a702-4b5d71600b2c">
After:
<img width="725" alt="image"
src="https://github.com/user-attachments/assets/c6c76e6a-d3fd-43dc-bfb0-b3a4a3fe2b6b">
## Summary
If a supported environment includes a Python marker, we don't simplify
it out, despite _storing_ the simplified markers. This PR modifies the
validation code to compare simplified to simplified markers.
Closes https://github.com/astral-sh/uv/issues/7876.
## Summary
When using `uv tree --package foo`, an extra empty line appears at the
beginning, which seems unnecessary since `uv tree` without the package
option doesn’t have this. It’s possible that the intention was to add
separation between packages, i.e. the correct implementation shoule be:
```rust
if !std::mem::take(&mut first) {
lines.push(String::new());
}
```
Even if corrected, this extra spacing might be redundant as `uv tree`
doesn’t include these empty lines between packages by default.
```console
$ uv init project
$ cd project
$ uv init foo
$ uv tree
Using CPython 3.12.5
Resolved 2 packages in 1ms
foo v0.1.0
project v0.1.0
$ uv tree --package project
Using CPython 3.12.5
Resolved 2 packages in 1ms
project v0.1.0
```
## Summary
Closes https://github.com/astral-sh/uv/issues/4931.
## Test Plan
Tried running the following commands locally to make sure that all cases
work:
```
unset PAGER
cargo run -- help venv
```
With no pager set, `uv` correctly finds `less` on the system as it did
before and passes the help output to it.
---
```
PAGER= cargo run -- help venv
```
This correctly prints out to stdout and does not use any pager.
---
```
PAGER=most cargo run -- help venv
```
This correctly opens the `most` pager as shown below:
<img width="1917" alt="Screenshot 2024-07-27 at 5 14 42 PM"
src="https://github.com/user-attachments/assets/dfaa5a83-b47e-4f5c-9be1-b0b1e9818932">
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
We now display the "Did you mean `python-dotenv`?"-style errors on build
failure, rather than in `uv add`. This is less opinionated and couples
us less to specific content in the registry.
## Test Plan
Would it be okay to expose this struct? We currently use our own
ResolveProvider, and it would be nice to use the `FlatDistributions` for
easy `VersionMap` creation.
Thanks!
## Summary
Create a function main as the default for a packaged app. Configure the
default executable as:
`example-packaged-app = "example_packaged_app:main"`
Which is often what you want - the executable has the same name as the
app.
The purpose is to more often hit what the user wants, so they don't have
to even rename the command to start developing.
## Test Plan
- existing tests are updated
## Summary
This PR enables users to provide multiple source entries in
`tool.uv.sources`, e.g.:
```toml
[tool.uv.sources]
httpx = [
{ git = "https://github.com/encode/httpx", tag = "0.27.2", marker = "sys_platform == 'darwin'" },
{ git = "https://github.com/encode/httpx", tag = "0.24.1", marker = "sys_platform == 'linux'" },
]
```
The implementation is relatively straightforward: when we lower the
requirement, we now return an iterator rather than a single requirement.
In other words, the above is transformed into two requirements:
```txt
httpx @ git+https://github.com/encode/httpx@0.27.2 ; sys_platform == 'darwin'
httpx @ git+https://github.com/encode/httpx@0.24.1 ; sys_platform == 'linux'
```
We verify (at deserialization time) that the markers are
non-overlapping.
Closes https://github.com/astral-sh/uv/issues/3397.
## Summary
This was brought up on Twitter recently. `dotenv` hasn't been updated in
years and doesn't build successfully anymore. Users almost always mean
to install `python-dotenv`. I think we can add helpful hints here to
point users in the right direction.
## Test Plan
## Summary
`click` has one dependency of `colorama` only on Windows, `uv tree
--invert` should not include `colorama` on non-Windows platforms, but
currently:
```console
$ uv init
$ uv add click
$ uv tree --invert --python-platform macos
colorama v0.4.6
```
it should:
```console
$ uv tree --invert --python-platform macos
click v8.1.7
└── project v0.1.0
```
## Summary
This PR modifies our parsing to allow spaces in URLs. I don't know if
this is a correct change... But we now parse URLs until we see:
- A newline.
- A semicolon (marker) or hash (comment), _preceded_ by a space. We
parse the URL until the last
non-whitespace character (inclusive).
- A semicolon (marker) or hash (comment) _followed_ by a space. We treat
this as an error, since
the end of the URL is ambiguous (e.g., `https://foo.com; marker`) would
be a URL that ends in `;`).
Closes https://github.com/astral-sh/uv/issues/6032.
## Summary
This is a longstanding piece of technical debt. After we resolve, we
have a bunch of `ResolvedDist` entries. We then convert those to
`Requirement` (which is lossy -- we lose information like "the index
that the package was resolved to"), and then back to `Dist`.
## Summary
Historically, we've allowed the use of wheels that were downloaded from
PyPI even when the user passes `--no-binary`, if the wheel exists in the
cache. This PR modifies the cache lookup code such that we respect
`--no-build` and `--no-binary` in those paths.
Closes https://github.com/astral-sh/uv/issues/2154.
## Summary
My last changes (#6616) used by mistake == instead of !=.
😥 Making values currently never trimmed despite
what we wanted.
Values should now be trimmed if needed.
Also removes the trim of the header name, because if a header contains
spaces, the header will be skipped by the mailparse crate in the first
place.
## Test Plan
- A unit test has been added to validate that we correctly trim values.
- A unit test has been added to validate the header names containing
spaces are skipped.
## Summary
I have a workflow where I want use `uv` as a dependency solver only, and
manage my environments with external tooling (Nix).
## Test Plan
Manually tested. Automated testing seems excessive for such a trivial
change.
## Problems
It's still not as useful as I'd like it to be.
`uv` uncondtionally creates a virtual environment, something I would
expect that `--no-sync` should disable.
This looks a bit more tricky to achieve and I'm not sure about how to
best structure it.
## Summary
This is another attempt using `module: bool` instead of `module:
Option<String>` following #7322.
The original PR can't be reopened after a force-push to the branch, I've
created this new PR.
Resolves#6638
## Summary
This PR fixes#7733. According to [CPython documentation on
`sys.stdout`](https://docs.python.org/3.12/library/sys.html#sys.stdout),
when `stdout`/`stderr` is non-character device like pipe, the encoding
will be set to system locale on windows. However, on the Rust side
`stdout_reader` and `stderr_reader` expect them to be encoded in UTF-8
and will fail when child process write non-ASCII character to
stdout/stderr, e.g., build directory name containing non-ASCII
character.
Both
[CPython3](https://docs.python.org/3.12/using/cmdline.html#envvar-PYTHONIOENCODING)
and [PyPy](https://doc.pypy.org/en/default/man/pypy3.1.html#environment)
support environment variable `PYTHONIOENCODING`. When it is set to
`utf-8`, python will use UTF-8 encoding for `stdin`/`stdout`/`stderr`.
Since `stdin` is not used by the spawned python process and we expect
`stdout`/`stderr` to use UTF-8, this fix should work as expected.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
I only tested it on my computer with CPython 3.12 and 3.7. With the fix
applied I confirmed that [the case I
described](https://github.com/astral-sh/uv/issues/7733#issuecomment-2380416093)
is fixed.
I'm using Windows 11 with system locale set to code page 936.
## Summary
Small follow up to https://github.com/astral-sh/uv/pull/7724
## Test Plan
`cargo test`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Resolves#7705
## Test Plan
`cargo test` and tested locally.
The snapshots were unstable due to the packages being built in a
non-deterministic order, so I used the quiet flag to suppress the
output.
Another question is whether we should label the build output to indicate
which package it belongs to?
## Summary
Adds a helpful context message when `uvx` is run without arguments
To clarify, it is displaying the installed tools.
This addresses confusion, such as the one highlighted in issue #7348,
by making the output more user-friendly and informative.
Related #4024
## Test Plan
Updated the test snapshots to include the new output.
Running the tests locally with `cargo nextest run` confirms that the
tests pass.
The CI pipeline should also pass.
### Manuel Testing
**uvx**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uvx
Provide a command to invoke with `uvx <command>` or `uvx --from <package> <command>`.
The following tools are already installed:
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
See `uvx --help` for more information.
```
**uv tool list**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uv tool list
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
```
**uv tool run**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uv tool run
Provide a command to invoke with `uv tool run <command>` or `uv tool run --from <package> <command>`.
The following tools are already installed:
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
See `uv tool run --help` for more information.
```
---
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
---------
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
## Summary
Similiar to `cargo init --vcs <VCS>`, this PR adds the `--vcs <VCS>`
flag for `uv init`, allowing to create a version control system during
initialization. By default, `uv init` will create a Git repository if
the `--vcs` flag is not provided. Use `--vcs none` to disable this
feature.
Currently, only Git is supported. While Cargo also supports hg, pijul,
and fossil, this initial PR only includes Git. We can add more later if
there are any user requests.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This PR adds support for ```uv init --script```, as defined in issue
#7402 (started working on this before I saw jbvsmo's PR). Wanted to
highlight a few decisions I made that differ from the existing PR:
1. ```--script``` takes a path, instead of a path/name. This potentially
leads to a little ambiguity (I can certainly elaborate in the docs,
lmk!), but strictly allowing ```uv init --script path/to/script.py```
felt a little more natural than allowing for ```uv init --script path/to
--name script.py``` (which I also thought would prompt more questions
for users, such as should the name include the .py extension?)
2. The request is processed immediately in the ```init``` method,
sharing logic in resolving which python version to use with ```uv add
--script```. This made more sense to me — since scripts are meant to
operate in isolation, they shouldn't consider the context of an
encompassing package should one exist (I also think this decision makes
the relative codepaths for scripts/packages easier to follow).
3. No readme — readme felt a little excessive for a script, but I can of
course add it in!
---------
Co-authored-by: João Bernardo Oliveira <jbvsmo@gmail.com>
uv will soon support both a build frontend (`uv build`) and a build
backend (`build-system = "uv"`). To avoid the name clash, I'm renaming
the `uv-build` crate to `uv-build-frontend`. In a follow-up PR, I will
add a `uv-build-backend` crate with the build backend implementation.
This PR adds support for upgrading the build environment of tools with
the addition of a ```--python``` argument to ```uv upgrade```, as
specified in #7471.
Some things to note:
- I added support for individual packages — I didn't think there was a
good reason for ```--python``` to only apply to all packages
- Upgrading with ```--python``` also upgrades the package itself — I
think this is fair as if a user wants to _strictly_ switch the version
of Python being used to build a tool's environment they can use ```uv
install```. This behavior can of course be modified if others don't
agree!
Closes https://github.com/astral-sh/uv/issues/6297.
Closes https://github.com/astral-sh/uv/issues/7471.
#7226 modified the check to skip prefetching of source dists without
proper minimum-version bounds, and wound up flipping the boolean
expression. This change flips the some/none expression so that the
intended skip happens as expected.
Fixes#7680.
Closes#7118
This only really affects managed interpreters, as we exclude alternative
Python implementations from the search path during the
`VersionRequest::executable_names` part of discovery.
## Summary
Random, but I noticed that we can remove a ton of serialize and
deserialize derives by using `rkyv` for the flat-index caches. (We
already use `rkyv` for these same structs in the registry cache.)
This PR adds some additional sanity checking on resolution graphs to
ensure we can never install different versions of the same package into
the same environment.
I used code similar to this to provoke bugs in the resolver before the
release, but it never made it into `main`. Here, we add the error
checking to the creation of `ResolutionGraph`, since this is where it's
most convenient to access the "full" markers of each distribution.
We only report an error when `debug_assertions` are enabled to avoid
rendering `uv` *completely* unusuable if a bug were to occur in a
production binary. For example, maybe a conflict is detected in a marker
environment that isn't actually used. While not ideal, `uv` is still
usable for any other marker environment.
Closes#5598
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
closes#4828
First iteration for an implementation. I need to add more tests but
wanted your opinion on the implementation first.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Currently tested using the following command but will add tests shortly:
```console
D:\repo\uv> cargo run venv -p 3.13t && .venv\Scripts\python.exe
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
Running `target\debug\uv.exe venv -p 3.13t`
Using Python 3.13.0rc1 interpreter at: C:\Users\bschoen\AppData\Local\Programs\Python\Python313\python3.13t.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e78, Jul 31 2024, 21:06:58) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
There are two parts to this.
The first is a restructuring and refactoring. We had some debt around
expected executable name generation, which we address here by
consolidating into a single function that generates a combination of
names. This includes a bit of extra code around free-threaded variants
because this was written on top of #7431 — I'll rebase that on top of
this.
The second addresses some bugs around alternative implementations.
Notably, `uv python list` does not discovery executables with
alternative implementation names. Now, we properly generate all of the
executable names for `VersionRequest::Any` (originally implemented in
https://github.com/astral-sh/uv/pull/7508) to properly show all the
implementations we can find:
```
❯ cargo run -q -- python list --no-python-downloads
cpython-3.12.6-macos-aarch64-none /opt/homebrew/opt/python@3.12/bin/python3.12 -> ../Frameworks/Python.framework/Versions/3.12/bin/python3.12
cpython-3.11.10-macos-aarch64-none /opt/homebrew/opt/python@3.11/bin/python3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/python3.11
cpython-3.9.6-macos-aarch64-none /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.9/bin/python3
pypy-3.10.14-macos-aarch64-none /opt/homebrew/bin/pypy3 -> ../Cellar/pypy3.10/7.3.17/bin/pypy3
```
While doing both of these changes, I ended up changing the priority of
interpreter discovery slightly. For example, given that the executables
are in the same directory, do we query `python` or `python3.10` first
when you request `--python 3.10`? Previously, we'd check `python3.10`
but I think that was an incorrect optimization. I think we should always
prefer the bare name (i.e. `python`) first. Similarly, this applies to
`python` and an executable for an alternative implementation like
`pypy`. If it's not compatible with the request, we'll skip it anyway.
We might have to query more interpreters with this approach but it seems
rare.
Closes https://github.com/astral-sh/uv/issues/7286 superseding
https://github.com/astral-sh/uv/pull/7508
- **Do not attempt to reflink directories on linux**
- **Refactor clone_recursive**
## Summary
On linux, reflink does not work on a directory. Currently, we first
attempt to reflink directory, and only if it fails with `AlreadyExists`
we attempt to reflink recursively.
This has the effect that, on linux, `uv pip install --link-mode=clone`
would always fall back to `copy`.
We resolve this by only attempting to reflink directories on macos. In
the process, we refactored `clone_recursive` in an attempt to make it
easier to reason about its logic.
## Test Plan
I tested that after this change, `uv pip install --link-mode=clone
numpy` would behave as expected in the following cases:
* linux, btrfs filesystem, venv on the same filesystem as cache
(correctly reflinked)
* linux, btrfs filesystem, venv on a different filesystem than cache
(fallback to copy)
I have not tested it on macos or windows, as I currently don't have
access to any macos or windows machines, unfortunately.
## Summary
If the `requires-python` bound expands, the space covered by
`resolution-markers` may no longer include all supported Python
versions. In such cases, we need to avoid reusing the forks (but we
_can_ reuse the preferred versions).
Closes https://github.com/astral-sh/uv/issues/7618.
## Summary
`uv run --project ./path/to/project` now uses the provided directory as
the starting point for any file discovery. However, relative paths are
still resolved relative to the current working directory.
Closes https://github.com/astral-sh/uv/issues/5613.
## Summary
`#[serde(flatten)]` has a disastrous effect on error messages: serde no
longer tells you which field errored, nor does it show it to you in the
diagnostic output.
Before:
```
warning: Failed to parse `pyproject.toml` during settings discovery:
TOML parse error at line 9, column 1
|
9 | [tool.uv]
| ^^^^^^^^^
invalid type: string "foo", expected a sequence
```
After:
```
warning: Failed to parse `pyproject.toml` during settings discovery:
TOML parse error at line 10, column 19
|
10 | extra-index-url = "foo"
| ^^^^^
invalid type: string "foo", expected a sequence
```
Closes https://github.com/astral-sh/uv/issues/7113.
## Summary
This reverts commit 3060fd22c0.
These are now _never_ shown to users, because `tracing` isn't set up at
that point. I'm going to try and improve the solution more holistically,
but this is better than the status quo.
Closes https://github.com/astral-sh/uv/issues/7573.
This enhances the hints generator in the resolver with some heuristic to
detect and warn in case of failures due to version mismatches on a local
package. Those may be the symptom of name conflict/shadowing with a
transitive dependency.
Closes: https://github.com/astral-sh/uv/issues/7329
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Because a problem was found with Powershell and combining the generated
completion scripts for uv and uvx, let's try separating uv and uvx
command completion scripts.
The generated powershell script template can be seen in clap_complete
source, and it starts with `using` directives, which makes it impossible
(apparently) to concatenate two of those script outputs.
As a side effect, this is available under `uv tool run
--generate-shell-completion` too.
Fixes#7482
## Test Plan
- `eval "$(cargo run --bin uvx -- --generate-shell-completion bash)"`
- Test Powershell
## Summary
The AGPL license is confusing some analyzers. This replaces pretix with
saleor which (similarly) is a web application.
Closes https://github.com/astral-sh/uv/issues/7566.
## Summary
`uv cache prune --ci` will remove the source distribution directory. If
we then need to build a _different_ wheel (e.g., you're building a
package that has Python minor version-specific wheels), we fail, because
we expect the source to be there.
Now, if the source is missing, we re-download it. It would be slightly
easier to just _ignore_ that revision, but that would mean we'd also
lose the already-built wheels -- so if you ran against many Python
versions, we'd continuously lose the cached data.
Closes https://github.com/astral-sh/uv/issues/7543.
## Test Plan
We can add tests, but they _need_ to build non-pure Python wheels, which
tends to be expensive...
For reference:
```console
$ cargo run venv --python 3.12
$ cargo run pip install mercurial==6.8.1 --verbose
$ cargo run cache prune --ci
$ cargo run venv --python 3.11
$ cargo run pip install mercurial==6.8.1 --verbose
```
I also did this with a local `.tar.gz` that I downloaded from PyPI.
## Summary
Both of these can contain rkyv data in their HTTP cache envelopes. As
such, the entries aren't readable by earlier versions of uv, and `uv
cache prune` can break. I should make `uv cache prune` robust to this,
but this feels safest.
## Summary
I think this is just inverted. It means that when we fail in
https://github.com/astral-sh/uv/issues/7553, we show a message for
"invalid Python implementation" (since there are some wheels that don't
match), but we should be showing "invalid platform", matching the order
of operations in our compatibility check.
Closes https://github.com/astral-sh/uv/issues/7553.
Adds display of the target path of the link (since the link filename
itself is basically static) and distinguishes between broken links and
missing files.
## Summary
Improve the description of override-dependencies based on the statement
in `concepts/resolution.md`: "As with constraints, overrides do not add
a dependency on the package and only take effect if the package is
requested in a direct or transitive dependency."
I tested it locally, `concepts/resolution.md` is correct. It would be
better to also include this in the Reference Chapter of the docs.
I think it's best practice to use a placeholder when we transform
something, and #7522 is having snapshot issues because this filter
conflicts with `with_filtered_virtualenv_bin`
As we support more complex Python discovery behaviors such as:
- #7431
- #7335
- #7300
`Any` is no longer accurate, we actually are looking for a reasonable
default Python version to use which may exclude the first one we find.
Separately, we need the idea of `Any` to improve behavior when listing
versions (e.g., #7286) where we do actually want to match _any_ Python
version. As a first step, we'll rename `Any` to `Default`. Then, we'll
introduce a new `Any` that actually behaves as we'd expect.
Recently, rkyv 0.8 was released. Its API is a fair bit simpler now for
higher level uses (like for us in `uv`) and results in us being able to
delete a fair bit of code. This also removes our last dependency on `syn
1.0`, and thus drops that dependency.
Performance (via testing on the `transformers` example) seems to remain
about the same, which is what was expected:
```
$ hyperfine -w5 -r100 'uv lock' 'uv-ag-rkyv-update lock'
Benchmark 1: uv lock
Time (mean ± σ): 55.6 ms ± 6.4 ms [User: 30.4 ms, System: 35.1 ms]
Range (min … max): 43.0 ms … 73.1 ms 100 runs
Benchmark 2: uv-ag-rkyv-update lock
Time (mean ± σ): 56.5 ms ± 7.2 ms [User: 30.5 ms, System: 36.3 ms]
Range (min … max): 39.1 ms … 71.5 ms 100 runs
Summary
uv lock ran
1.02 ± 0.18 times faster than uv-ag-rkyv-update lock
```
Closes#7415
This changes the structure of the hints generator in the resolver when
encountering solution errors, so that it re-uses a single output buffer
owned by the caller.
It avoids repeated allocations of a temporary buffer within each
recursive function call.
## Summary
See: https://github.com/astral-sh/uv/issues/7485. The test was using `uv
pip sync` which doesn't require fetching metadata, and the failure was
in fetching metadata.
## Summary
Closes https://github.com/astral-sh/uv/issues/7485.
## Test Plan
```
$ cargo run cache clean
$ cargo run venv
$ cargo run pip install django-allauth==0.51.0
$ cargo run venv
$ cargo run pip install django-allauth==0.51.0
```
This changes `uv tool install` behavior with regards to re-using
existing environments.
In particular, this replaces the existing version-matching logic with a
tighter one, enforcing
a same-interpreter match.
This allows to properly switch between system and managed interpreter,
at the cost of
more eagerly invalidating existing environments every time there is an
interpreter change.
Closes: https://github.com/astral-sh/uv/issues/7320
## Summary
This PR enables users to provide pre-defined static metadata for
dependencies. It's intended for situations in which the user depends on
a package that does _not_ declare static metadata (e.g., a
`setup.py`-only sdist), and that is expensive to build or even cannot be
built on some architectures. For example, you might have a Linux-only
dependency that can't be built on ARM -- but we need to build that
package in order to generate the lockfile. By providing static metadata,
the user can instruct uv to avoid building that package at all.
For example, to override all `anyio` versions:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]
[[tool.uv.dependency-metadata]]
name = "anyio"
requires-dist = ["iniconfig"]
```
Or, to override a specific version:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]
[[tool.uv.dependency-metadata]]
name = "anyio"
version = "3.7.0"
requires-dist = ["iniconfig"]
```
The current implementation uses `Metadata23` directly, so we adhere to
the exact schema expected internally and defined by the standards. Any
entries are treated similarly to overrides, in that we won't even look
for `anyio@3.7.0` metadata in the above example. (In a way, this also
enables #4422, since you could remove a dependency for a specific
package, though it's probably too unwieldy to use in practice, since
you'd need to redefine the _rest_ of the metadata, and do that for every
package that requires the package you want to omit.)
This is under-documented, since I want to get feedback on the core ideas
and names involved.
Closes https://github.com/astral-sh/uv/issues/7393.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
close#6272
## Test Plan
<!-- How was it tested? -->
As in https://github.com/astral-sh/uv/pull/6262
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
When syncing a lockfile, we need to respect credentials defined in the
`pyproject.toml`, even if they won't be used for resolution.
Unfortunately, this includes credentials in `tool.uv.sources`,
`tool.uv.dev-dependencies`, `project.dependencies`, and
`project.optional-dependencies`.
Closes https://github.com/astral-sh/uv/issues/7453.
## Summary
This PR adds support to include Python pre-releases when requesting
versions.
Check out the docs for commands that support the `Python` option:
```text
--python, -p python
The Python interpreter to use for the virtual environment.
```
At least the following scenarios are supported:
```bash
3.13.0a1
3.13b2
3.13rc4
313rc1
```
## Test Plan
I added a basic unit test to `uv/crates/uv-python/src/discovery.rs`. I
could have added more, but I have not discovered any relevant places.
CI passes
Note: I was unable to execute the entire test set locally. There were at
least some timeout issues (some tests took over 60 seconds).
========== output ===========
beta version
```bash
cargo run -- venv --python 3.13.0b3 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.20s
Running `target/debug/uv venv --python 3.13.0b3`
Using Python 3.13.0b3 interpreter at: /home/mikko/.pyenv/versions/3.13.0b3/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
````
release candidate
```bash
cargo run -- venv --python 3.13.0rc2 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.83s
Running `target/debug/uv venv --python 3.13.0rc2`
Using Python 3.13.0rc2 interpreter at: /home/mikko/.pyenv/versions/3.13.0rc2/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
```bash
cargo run -- venv --python 313rc2 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.31s
Running `target/debug/uv venv --python 313rc2`
Using Python 3.13.0rc2 interpreter at: /home/mikko/.pyenv/versions/3.13.0rc2/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Generate shell completion for uvx.
Create a `uvx` toplevel command just for completion by combining `uv
tool uvx` (hidden alias for `uv tool run`) with global arguments. This
explicit combination is needed otherwise global arguments are missing
(if they are missing, clap debug assertions fail when `uv tool run`
arguments refer to global arguments in directives like conflicts with).
Fixes#7258
## Test Plan
- Tested using bash using `eval "$(cargo run --bin uv
generate-shell-completion bash)"`
## Summary
All the registry wheels were getting cached under
`index/b2a7eb67d4c26b82` rather than `pypi`, because we used
`IndexUrl::Url` rather than `IndexUrl::from`.
## Summary
It's very unlikely that retaining these is beneficial, since you tend to
partition the cache by platform anyway.
Closes https://github.com/astral-sh/uv/issues/7394.
## Summary
Since https://github.com/astral-sh/uv/pull/7208, this is now _always_
firing, for every directory, because the version gets normalized (e.g.,
`1.2.3` gets normalized to `1-2-3`, which never matches the parsed
version). pip doesn't warn here, I guess we won't either, because I
can't figure out a robust way to do this... We need to get the
non-normalized remainder after stripping the normalized package name,
but we strip the normalized package name from the normalized string, so
we only have a normalized remainder.
## Summary
Running `uv lock --no-sources` should still include dev dependencies,
since dev dependencies are defined separately from sources.
Closes https://github.com/astral-sh/uv/issues/7406.
We got user reports where users were confused about why they can't use
`[project.urls]` in `pyproject.toml` (i think that's from poetry?). This
PR adds a hint that (according to PEP 621), you need to set
`project.name` when using any `project` fields. (PEP 621 also requires
`project.version` xor `dynamic = ["version"]`, but we check that later.)
The intermediate parsing layer to tell apart syntax errors from schema
errors doesn't incur a performance penalty according to epage
(https://github.com/toml-rs/toml/issues/778#issuecomment-2310369253).
Closes#6419Closes#6760
closes#7365
Summary
This pull request adds support for additional file extension aliases in
the SourceDistExtension and ExtensionError enums. The newly supported
file extensions include .tbz, .tgz, .txz, .tar.lz, .tar.lzma. These
changes align the extensions supported by the SourceDistExtension with
those used in Python packaging tools, enhancing compatibility with a
broader range of source distribution formats.
Test Plan
should be added or updated to verify that the new extensions are
correctly recognized as valid source distributions and that errors are
correctly raised when unsupported extensions are provided.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Part of https://github.com/astral-sh/uv/issues/7007.
Settings documentation reference currently doesn't separate "project
metadata" and "configuration" options, implying that it's possible to
set things like `dev-dependencies` in `uv.toml` while it's not. This is
an attempt at better separating those options, by having 2 different
sections:
- `Project metadata`, that holds configuration that can only be set in
`pyproject.toml`
- `Configuration`, that holds configuration that can be set both in
`pyproject.toml` and `uv.toml`
Here are some screenshots to show what this looks like (note that I
don't have code highlighting in the right navigation, which makes them
clunky, as first item is always bigger because of the missing "span" --
I think that's because it's an `mkdocs-material` insider feature, since
I have the same thing on `main` branch):
- Right side navigation:
<img width="241" alt="Screenshot 2024-09-05 at 01 19 50"
src="https://github.com/user-attachments/assets/012f64a4-8d34-4e34-a506-8d02dc1fbf98">
<img width="223" alt="Screenshot 2024-09-05 at 01 20 01"
src="https://github.com/user-attachments/assets/0b0fb71d-c9c3-4ee3-8f6e-cf35180b1a99">
- An option from "Project metadata" section that only applies to
`pyproject.toml`:
<img width="788" alt="Screenshot 2024-09-05 at 01 20 11"
src="https://github.com/user-attachments/assets/64349fbb-8623-4b81-a475-d6ff38c658f1">
- An option from "Configuration" section that applies both to
`pyproject.toml` and `uv.toml`:
<img width="787" alt="Screenshot 2024-09-05 at 01 20 33"
src="https://github.com/user-attachments/assets/732e43d3-cc64-4f5a-8929-23a5555d4c53">
## Test Plan
Local run of the documentation.
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Updates the output of `uv export` to include the command that produced
it, similar to how `uv pip compile` does. This addresses #7159 - I had
this same itch today, figured it was a good time to dive in!
## Test Plan
All the export unit tests were updated to test the new output format.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
fix symbol error
## Test Plan
<!-- How was it tested? -->
Signed-off-by: liangmulu <liangmulu@outlook.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
It often reaches the GitHub API rate limit and shows error like `error:
HTTP status client error (403 Forbidden) for url
(https://api.github.com/repos/astral-sh/uv/releases)` when running `uv
self update`.
To bypass this rate limit issue, allow user to pass a GitHub token via
`--token` or `UV_GITHUB_TOKEN` env.
## Test Plan
<!-- How was it tested? -->
---------
Signed-off-by: Frost Ming <me@frostming.com>
Summary
This pull request fixes a typo in the --build-constraints flag, which
should be singular (--build-constraint). This update ensures consistency
across the documentation and prevents potential confusion for users.
Closes#7315
## Test Plan
The change was verified by reviewing the relevant documentation files
where the flag is referenced. No functional code changes were made, so
no additional testing is required beyond confirming the documentation
update.
## Tested
The change was tested by visually inspecting the updated documentation
to confirm that the typo has been corrected
## Summary
This is arguably breaking, arguably a bug... Today, if project A depends
on project B, and you install A with dev dependencies enabled, you also
get B's dev dependencies. I think this is incorrect. Just like you
shouldn't be importing B's dependencies from A, you shouldn't be using
B's dev dependencies when developing on A.
Closes#7310.
Similar to our semantics for packages with pre-release versions.
We will not use prerelease versions unless there are only prerelease
versions available, a specific version is requested,
or the prerelease version is found in a reasonable source (active
environment, explicit path, etc. but not `PATH`).
For example, `uv python install 3.13 && uv run python --version` will no
longer use `3.13.0rc2` unless that is the only Python version available,
`--python 3.13` is used, or that's the Python version that is present in
`.venv`.
## Summary
This has been asked for a few times. There are risks that these checks
could be slow, but they're buyer-beware.
Closes https://github.com/astral-sh/uv/issues/7246.
## Summary
We have to call `to_dist` to get metadata while validating the lockfile,
but some of the distributions won't match the current platform -- and
that's fine!
## Summary
We need to apply the `--no-install` filters earlier, such that we don't
error if we only have a source distribution for a given package when
`--no-build` is provided but that package is _omitted_.
Closes#7247.
Following #7263 the 3.13.0rc2 releases are at the top of the download
list but we should not select them unless 3.13 is actually requested.
Prior to this, `uv python install` would install `3.13.0rc2`.
```
❯ cargo run -- python install --no-config
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv python install --no-config`
Searching for Python installations
Installed Python 3.12.6 in 1.33s
+ cpython-3.12.6-macos-aarch64-none
```
```
❯ cargo run -- python install --no-config 3.13
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv python install --no-config 3.13`
Searching for Python versions matching: Python 3.13
Installed Python 3.13.0rc2 in 1.18s
+ cpython-3.13.0rc2-macos-aarch64-none
```
## Summary
Replace the unmaintained `tokio-tar` crate with the `krata-tokio-tar`
fork. The latter just merged a fix necessary for the crate to work on
PowerPC, and has better chances of future maintenance.
Fixes#3423
## Test Plan
`cargo test`
This is preparatory work for the upload functionality, which needs to
read the METADATA file and attach its parsed contents to the POST
request: We move finding the `.dist-info` from `install-wheel-rs` and
`uv-client` to a new `uv-metadata` crate, so it can be shared with the
publish crate.
I don't properly know if its the right place since the upload code isn't
ready, but i'm PR-ing it now because it already had merge conflicts.
## Summary
If `--config-settings` are provided, we cache the built wheels under one
more subdirectory.
We _don't_ invalidate the actual source (i.e., trigger a re-download) or
metadata, though -- those can be reused even when `--config-settings`
change.
Closes https://github.com/astral-sh/uv/issues/7028.
Let's promote type hints!
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
The generated script now annotates the return type of the dummy function
`hello()`.
## Test Plan
<!-- How was it tested? -->
All existing tests have been synced with this update.
## Summary
This PR adds a more flexible cache invalidation abstraction for uv, and
uses that new abstraction to improve support for dynamic metadata.
Specifically, instead of relying solely on a timestamp, we now pass
around a `CacheInfo` struct which (as of now) contains
`Option<Timestamp>` and `Option<Commit>`. The `CacheInfo` is saved in
`dist-info` as `uv_cache.json`, so we can test already-installed
distributions for cache validity (along with testing _cached_
distributions for cache validity).
Beyond the defaults (`pyproject.toml`, `setup.py`, and `setup.cfg`
changes), users can also specify additional cache keys, and it's easy
for us to extend support in the future. Right now, cache keys can either
be instructions to include the current commit (for `setuptools_scm` and
similar) or file paths (for `hatch-requirements-txt` and similar):
```toml
[tool.uv]
cache-keys = [{ file = "requirements.txt" }, { git = true }]
```
This change should be fully backwards compatible.
Closes https://github.com/astral-sh/uv/issues/6964.
Closes https://github.com/astral-sh/uv/issues/6255.
Closes https://github.com/astral-sh/uv/issues/6860.
## Summary
We now track the discovered `IndexCapabilities` for each `IndexUrl`. If
we learn that an index doesn't support range requests, we avoid doing
any batch prefetching.
Closes https://github.com/astral-sh/uv/issues/7221.
## Summary
We were only applying exclusions when discovering the root, apparently.
Our logic now matches the original intent, which is...
- `exclude` always post-filters `members`.
- We don't treat globs any differently than non-globs.
The one confusing setup that falls out of this is that given:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar"]
```
`foo/bar/baz` **would** be included. To exclude it, you would need:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar/*"]
```
Closes https://github.com/astral-sh/uv/issues/7071.
## Summary
If we have a singleton `Range`, we don't need to iterate over the map of
available ranges; instead, we can just get the singleton directly.
Closes#6131.
## Summary
Use a path file (`.pth`) instead of `sitecustomize.py` for configuring
path in emphemeral virtualenvs, overlaying the ephemeral venv on top of
the base `.venv`.
`sitecustomize.py` is a module in the python installation and as such a
unique resource - homebrew pythons on macos already install such a file
and thus uv's `sitecustomize.py`, placed in the ephemeral env, did not
have any effect.
I don't find any documentation explicitly saying that addsitedir is
valid in `.pth` files but from trial it seems to be - and there is the
precedent of the existing _virtualenv.pth _virtualenv.py pair that do
nontrivial operations.
## Test Plan
- Testing on ephemeral venv, resolving to base venv including editable
install in base: done (py3.7, 3.12)
- Testing on homebrew python/macos: done (py3.11)
- tests: run_editable
Fixes#7152
## Summary
Fixes#7081
Treats source distribution `.tgz` the same as `.tar.gz` plans
## Test Plan
Quick Version
```bash
cd $(mktemp -d)
uv init
uv add --dev build
.venv/bin/python -m build -s .
mv -v dist/*tar.gz dist/"$(basename dist/*.tar.gz .tar.gz)".tgz
uv pip install dist/*.tgz
```
Can add a proper test to the branch if requested
Change the registry Python sorting implementation to be easier to
follow, making it clearer what it does and that it is a total order. No
functional changes.
## Summary
Explicitly list the formats and extensions that uv supports, based on
[this
list](86ee8d2c01/crates/distribution-filename/src/extension.rs (L70-L77)).
Not a huge fan of adding the section in `concepts/resolution.md`, but I
did not find a better place. Alternatively we could maybe add a
dedicated page that shortly explains Python package types (wheels,
sdists), where such a section could live?
## Test Plan
Local run of the documentation.
This finally gets rid of our hack for working around "hidden"
state. We no longer do a roundtrip marker serialization and
deserialization just to avoid the hidden state.
This adds new routines to `MarkerTree` for "simplifying" and
"complexifying" a tree with respect to lower and upper Python version
bounds.
In effect, "simplifying" a marker is what you do when you write it to a
lock file. Namely, since `uv.lock` includes a `requires-python` bound at
the top, one can say that it acts as a bound on the supported Python
versions. That is, it establishes a context in which one can assume that
bound is true. Therefore, the markers we write can be simplified using
this assumption.
The reverse is "complexifying" a marker, and it's what you do when you
read a marker from the lock file. Namely, once a marker is read, it can
be very difficult in code to keep the corresponding requires-python
context from the lock file. If you lose track of it and decide to
operate on the "simplified" marker, then it's trivial for that to
produce an incorrect result.
I split this change into its own commit because I'm hoping it
crystalizes what it means when we say "a `MarkerTree` has hidden state."
That is, it isn't so much that there is some explicit member of a
`MarkerTree` that is omitted, but rather, the lower and upper version
bounds on `python_full_version` are are rewritten as "unbounded" when
traversing the ADD for display.
We will actually retain this functionality, but rejigger it so that it's
explicit when we do this. In particular, this simplification has been
problematic for us because it fundamentally changes the truth tables of
a marker expression *unless* you are extremely careful to interpret it
only under the original context in which it was simplified. This is
quite difficult to do generally, and in prior work in #6268, we
completed a refactor where we worked around this type of simplification
and moved it to the edges of uv.
In subsequent commits, we'll re-implement this form of simplification as
a more explicit step.
## Summary
I think a better tradeoff here is to skip fetching metadata, even though
we can't validate the extras.
It will help with situations like
https://github.com/astral-sh/uv/issues/5073#issuecomment-2334235588 in
which, otherwise, we have to download the wheels twice.
(This is part of #5711)
## Summary
@BurntSushi and I spotted that the `derivative` crate is only used for
one enum in the entire codebase — however, it's a proc macro, and we pay
for the cost of (re)compiling it in many different contexts.
This replaces it with a private `Inner` core which uses the regular std
derive macros — inlining and optimizations should make this equivalent
to the other implementation, and not too hard to maintain hopefully
(versus a manual impl of `PartialEq` and `Hash` which have to be kept in
sync.)
## Test Plan
Trust CI?
This PR revives #6129, but is less bold:
* It doesn't rename anything. (I think the rename is probably right
though.)
* It doesn't change the _default_ `Debug` impl. Instead, it offers this
as a new `MarkerTree::debug_graph` method.
I found this pretty useful for debugging since it gives a display format
that is more faithful to the internal representation of a `MarkerTree`.
So I think it's worth having around. But making it available in `Debug`
is perhaps a bridge too far since it isn't as familiar as the typical
PEP 508 representation and isn't as succinct.
I did consider printing this when using `{:#?}` (i.e., the "alternate"
debug representation), but too many things use that (like `insta` I
think) to make it practical.
Closes#6129
## Summary
This has bothered me for a while and should be fairly impactful for
users. It requires a weird implementation, since the
distribution-building crate depends on the cache, and so the prune
operation can't live in the cache, since it needs to access internals of
the distribution-building crate.
Closes https://github.com/astral-sh/uv/issues/7096.
## Summary
Like `uv sync`, you can omit the current project (`--no-emit-project`),
a specific package (`--no-emit-package`), or the entire workspace
(`--no-emit-workspace`).
Closes https://github.com/astral-sh/uv/issues/6960.
Closes#6995.
Follow-up to #6959 and #6961: Use the reachability computation instead
of `propagate_markers` everywhere.
With `marker_reachability`, we have a function that computes for each
node the markers under which it is (`requirements.txt`, no markers
provided on installation) or can be (`uv.lock`, depending on the markers
provided on installation) included in the installation. Put differently:
If the marker computed by `marker_reachability` is not fulfilled for the
current platform, the package is never required on the current platform.
We compute the markers for each package in the graph, this includes the
virtual extra packages and the base packages. Since we know that each
virtual extra package depends on its base package (`foo[bar]` implied
`foo`), we only retain the base package marker in the `requirements.txt`
graph.
In #6959/#6961 we were only using it for pruning packages in `uv.lock`,
now we're also using it for the markers in `requirements.txt`.
I think this closes#4645, CC @bluss.
## Summary
We need to prioritize hashes for the distribution over hashes for the
related packages.
I think this needs to be redone entirely though. I can see other issues
with the current approach.
Closes https://github.com/astral-sh/uv/issues/7059.
## Summary
With #6917, there are a lot more PyPy downloads in `uv python list
--all-versions`. I find it clearer to have all the CPython downloads
listed, then all the PyPy downloads, rather than interspersing them. But
this is subjective, feel free to push back!
## Summary
This PR adds `--package` support to `uv build`, such that you can use
`--package` from anywhere in a workspace to build any member.
If a source directory is provided, we use that as the workspace root.
If a file is provided, we error.
For now, `uv build` only builds the current package, making it
semantically identical to `uv sync`.
## Summary
This PR allows users to run `uv build --wheel ./path/to/source.tar.gz`
to build a wheel from a source distribution. This is also the default
behavior if you run `uv build ./path/to/source.tar.gz`. If you pass
`--sdist`, we error.
## Summary
This PR exposes uv's PEP 517 implementation via a `uv build` frontend,
such that you can use `uv build` to build source and binary
distributions (i.e., wheels and sdists) from a given directory.
There are some TODOs that I'll tackle in separate PRs:
- [x] Support building a wheel from a source distribution (rather than
from source) (#6898)
- [x] Stream the build output (#6912)
Closes https://github.com/astral-sh/uv/issues/1510
Closes https://github.com/astral-sh/uv/issues/1663.
In the `lock_redact_https` test specifically, it prompts a link mode
warning from `uv` on my system. Debugging seems to suggest it is
provoked by attempting to hardlink between `/tmp` and `~/.local`. Since
these are on different file systems for me (with `/tmp` being a
ramdisk), it provokes the warning, and this turn spoils the snapshot
when running tests locally.
This PR adds a test specific filter rule to fix this.
## Summary
The error handlers now happen one level higher, matching on _any_ `Err`
that's returned from the lock-and-sync operations.
Closes https://github.com/astral-sh/uv/issues/7011.
`_virtualenv.py` doesn't need to import `__future__.annotations`, as it
has none.
Removing the import:
* Restores the action of the VIRTUALENV_PATCH on Python 3.6
* Eliminates 24 lines of error messages displayed by Python 3.6 when it
starts in an environment created by uv:
```plaintext
Error processing line 1 of /tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.pth:
Traceback (most recent call last):
File "~/.pyenv/versions/3.6.15/lib/python3.6/site.py", line 168, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "/tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.py", line 3
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
Remainder of file ignored
```
(Python displays the errors above twice.)
I appreciate the Python team no longer support Python 3.6, but
RedHat-style Linux distributions will support Python 3.6 in their
`/usr/libexec/platform-python` until [releasever 8 expires in
2029](https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide).
I'm happy for the community to move on, in general, but don't see the
harm in helping those who can't.
I'm not yet sure what in the “remainder of file ignored” is necessary
for my project's build, as I haven't yet finished digging that from
under Hatch. I'll follow up on #6426 when I do, so we can concentrate on
getting to the happy cow.
## Test Plan
```sh
( set -eu
export VIRTUAL_ENV="$(mktemp -d)"
./target/release/uv venv "$VIRTUAL_ENV" --python=python3.6
./target/release/uv pip install cowsay
$VIRTUAL_ENV/bin/python -m cowsay --text 'Look, a talking cow!' )
```
Happy output:
```plaintext
Using Python 3.6.15 interpreter at: ~/.local/bin/python3.6
Creating virtualenv at: /tmp/tmp.VHl4XNi3oI
Activate with: source /tmp//tmp.VHl4XNi3oI/bin/activate
Resolved 1 package in 929ms
Installed 1 package in 17ms
+ cowsay==6.0
____________________
| Look, a talking cow! |
====================
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Resolves issues mentioned in comments
* https://github.com/astral-sh/uv/issues/6699#issuecomment-2322515962
* https://github.com/astral-sh/uv/issues/6866#issuecomment-2322785906
Further investigation on the comments revealed that the pointer
arithmethic being performed in `let handle_start = unsafe {
crt_magic.offset(1 + handle_count) };` from [posy
trampoline](dda22e6f90/src/trampolines/windows-trampolines/posy-trampoline/src/bounce.rs (L146))
had some slight errors. Since `crt_magic` was a `*const u32`, doing an
offset by `1 + handle_count` would offset by too much, with some
possible out of bounds reads or attempts to call CloseHandle on garbage.
We needed to offset differently since we want to offset by
`handle_count` bytes after the initial offset as seen in
[launcher.c](888c48b568/PC/launcher.c (L578)).
Similarly, we needed to skip the first 3 handles, otherwise we'd still
be attempting to close standard I/O handles of the parent (in this case
the shell from `busybox.exe sh -l`).
I also added a few extra checks available from `launcher.c` which checks
if the handle value is `-2` just to match the distlib implementation
more closely and minimize differences.
## Test Plan
Manually compiled distlib's launcher with additional logging and
replaced `Lib/site-packages/pip/_vendor/distlib/t64.exe` with the
compiled one to log pointers. As a result, I was able to verify the
retrieved handle memory addresses in this function actually match in
both uv and distlib's implementation from within busybox.exe nested
shell where this behavior can be observed and manually tested.
I was also able to confirm this fixes the issues mentioned in the
comments, at least with busybox's shell, but I assume this would fix the
case with cmake.
## Open areas
`launcher.c` also [checks the
size](888c48b568/PC/launcher.c (L573-L576))
of `cbReserved2` before retrieving `handle_start` which this function
currently doesn't do. If we wanted to, we could add the additional check
here as well, but I wasn't fully sure why it wasn't added in the first
place. Thoughts?
```rust
// Verify the buffer is large enough
if si.cbReserved2 < (size_of::<u32>() as isize + handle_count + size_of::<HANDLE>() as isize * handle_count) as u16 {
return;
}
```
---------
Co-authored-by: konstin <konstin@mailbox.org>
When a package is included under a platform-specific marker, we know
that wheels that mismatch this marker can never be installed, so we drop
them from the lockfile.
In transformers, we have:
* `tensorflow-text`: `tensorflow-macos; python_full_version >= '3.13'
and platform_machine == 'arm64' and platform_system == 'Darwin'`
* `tensorflow-macos`: `tensorflow-cpu-aws; (python_full_version < '3.10'
and platform_machine == 'aarch64' and platform_system == 'Linux') or
(python_full_version >= '3.13' and platform_machine == 'aarch64' and
platform_system == 'Linux') or (python_full_version >= '3.13' and
platform_machine == 'arm64' and platform_system == 'Linux')`
* `tensorflow-macos`: `tensorflow-intel; python_full_version >= '3.13'
and platform_system == 'Windows'`
This means that `tensorflow-cpu-aws` and `tensorflow-intel` can never be
installed, and we can drop them from the lockfile.
## Summary
I'm not convinced that the behavior is correct as-implemented. When the
user passes a `--python >=3.8` or we discover a `requires-python` from
the workspace, we're currently writing that request out to
`.python-version`. I would probably rather that we write the resolved
patch version?
Closes https://github.com/astral-sh/uv/issues/6821.
The key change here is to use raw strings so that we don't need to
double-escape things like `\d`. And in particular, we rely on the fact
that `"\n"` and `r"\n"` are precisely equivalent when fed to
`Regex::new` in the `regex` crate.
Previously we were using `[+-~]`, but this includes the full range of
characters from `+` to `~`. Incidentally, this does include `-`. We
instead rewrite this as `[-+~]`, which probably matches the intent.
Unlike the previous update, this message is specifically referring to a
fork's markers inside the resolver. We probably *could* massage the
message to be simplified with respect to requires-python, but it's not
obvious to me that that is the right thing to do.
It is not clear whether this update is correct or not. Moreover, it's
not clear whether the status quo is correct or not. The problem is that
`transformers` is so big that it's very hard to understand what the
right output is without a deeper investigation.
One thing that is interesting is if fork prioritization is removed in
this PR *and* on `main`, then the differences in this ecosystem test go
away.
We've decided for now to move forward with this update even though we're
uncertain because this PR fixes a few outstanding correctness issues.
This update changes the error message to one that is worse than the
status quo, but it is still correct because `datasets >= 2.19` doesn't
actually exist given our `EXCLUDE_NEWER` in tests at present.
The underlying cause here seems to be in how PubGrub deals with
reporting incompatibilities. Namely, when it has `foo < 1` and
`foo >= 1`, it reports an incompatibility immediately before looking for
versions. But when it has `foo < 1` and `foo >= 1 ; marker`, then
because they aren't both pubgrub "packages," it starts requesting
versions first and hits the "not available" error path instead of the
"incompatible" error path.
Since this is more of an underlying issue with how we setup
`PubGrubPackage` and our interaction with pubgrub, we ended up deciding
to move forward here with the regression since this PR is fixing a
correctness issue. In particular, if one changes the `requires-python`
to `>=3.8`, then both `main` and this PR produce similarly bad error
messages.
This commit refactors how deal with `requires-python` so that instead of
simplifying markers of dependencies inside the resolver, we do it at the
edges of our system. When writing markers to output, we simplify when
there's an obvious `requires-python` context. And when reading markers
as input, we complexity markers with the relevant `requires-python`
constraint.
When I first wrote this routine, it was intended to only emit a trace
for the final "unioned" resolution. But we actually moved that semantic
operation to the construction of the resolution *graph*. So there is no
unioned `Resolution` any more.
But this is still useful to see. So I changed this to just emit a trace
of *every* resolution right before constructing the graph.
It might be nice to also emit a trace of the unioned graph too. Or
perhaps we should do that instead if this proves too noisy. (Although
this is only emitted at TRACE level.)
These are regression tests for #6269, #6412 and #6836. In this commit,
their test outputs are all wrong. We'll update these snapshots after
fixing the underlying bug by refactoring how `requires-python`
simplification works.
- Respect `UV_PROJECT_ENVIRONMENT` when in project root
- Add `--no-project` and `--no-workspace` to opt-out of above and
`requires-python` detection
- Rename `[NAME]` to `[PATH]` in CLI
Allows configuration of the (currently hard-coded) path to the virtual
environment in projects using the `UV_PROJECT_ENVIRONMENT` environment
variable.
If empty, we'll ignore it. If a relative path, it will be resolved
relative to the workspace root. If an absolute path, we'll use that.
This feature targets use in Docker images and CI. The variable is
intended to be set once in an isolated system and used for all uv
operations.
We do not expose a CLI option or configuration file setting — we may
pursue those later but I see them as lower priority. I think a
system-level environment variable addresses the most pressing use-cases
here.
This doesn't special-case the system environment. Which means that you
can use this to write to the system Python environment. I would
generally strongly recommend against doing so. The insightful comment
from @edmorley at
https://github.com/astral-sh/uv/issues/5229#issuecomment-2312702902
provides some context on why. More generally, `uv sync` will remove
packages from the environment by default. This means that if the system
environment contains any packages relevant to the operation of the
system (that are not dependencies of your project), `uv sync` will break
it. I'd only use this in Docker or CI, if anywhere. Virtual environments
have lots of benefits, and it's only [one line to "activate"
them](https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment).
If you are considering using this feature to use Docker bind mounts for
developing in containers, I would highly recommend reading our [Docker
container development
documentation](https://docs.astral.sh/uv/guides/integration/docker/#developing-in-a-container)
first. If the solutions there do not work for you, please open an issue
describing your use-case and why.
We do not read `VIRTUAL_ENV` and do not have plans to at this time.
Reading `VIRTUAL_ENV` is high-risk, because users can easily leave an
environment active and use the uv project interface today. Reading
`VIRTUAL_ENV` would be a breaking change. Additionally, uv is
intentionally moving away from the concept of "active environments" and
I don't think syncing to an "active" environment is the right behavior
while managing projects. I plan to add a warning if `VIRTUAL_ENV` is
set, to avoid confusion in this area (see
https://github.com/astral-sh/uv/pull/6864).
This does not directly enable centrally managed virtual environments. If
you set `UV_PROJECT_ENVIRONMENT` to an absolute path and use it across
multiple projects, they will clobber each other's environments. However,
you could use this with something like `direnv` to achieve "centrally
managed" environments. I intend to build a prototype of this eventually.
See #1495 for more details on this use-case.
Lots of discussion about this feature in:
- https://github.com/astral-sh/rye/issues/371
- https://github.com/astral-sh/rye/pull/1222
- https://github.com/astral-sh/rye/issues/1211
- https://github.com/astral-sh/uv/issues/5229
- https://github.com/astral-sh/uv/issues/6669
- https://github.com/astral-sh/uv/issues/6612
Follow-ups:
- #6835
- https://github.com/astral-sh/uv/pull/6864
- Document this in the project concept documentation (can probably
re-use some of this post)
Closes https://github.com/astral-sh/uv/issues/6669
Closes https://github.com/astral-sh/uv/issues/5229
Closes https://github.com/astral-sh/uv/issues/6612
## Summary
Http headers are supposed to be case-insensitive (RFC 2616), but there
are some implementations that don't normalize them.
I noticed it while migrating to `uv`, calls to an internal registry
failed. A man in the middle server helped me to find that `pip` uses
Title-Case while `uv pip` uses lowercase.
## Test Plan
I tested `uv` with the same server and now it works fine.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Separate exceptions for different timeouts to make it easier to debug
issues like #6105.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
Not tested at all.
## Summary
Closes#6319.
## Test Plan
I tested with `file:///mirror`, `file://localhost/mirror`, and
`http://mirror` to confirm that it was working as expected.
``` shell-session
/private/tmp/mirror-local 07:08:18
:) tree mirror
mirror/
└── 20240814/
└── cpython-3.12.5+20240814-aarch64-apple-darwin-install_only_stripped.tar.gz
```
<img width="626" alt="image"
src="https://github.com/user-attachments/assets/9c04224d-305c-47ee-a524-4a6abeb79da4">
## Summary
Right now, we have slightly different `requires-python` semantics for
`-p 3.11` vs. `-p 3.11 --universal`, and slightly different (wrong)
semantics for how we compare against the _installed_ Python version
(which doesn't ignore upper bounds, but should).
This PR rips it all out and replaces it with consistent semantics across
`uv lock`, `uv pip compile -p 3.11`, and `uv pip compile -p 3.11
--universal`. We now always ignore upper bounds.
Closes https://github.com/astral-sh/uv/issues/6859.
Closes https://github.com/astral-sh/uv/issues/5045.
Forward an error for missing temp directories:
```
$ env TMPDIR=.tmp uv-debug pip install httpx
error: No such file or directory (os error 2) at path "/home/konsti/projects/uv/.tmp/.tmpgIBhhh"
```
Fixes#6878
## Summary
A few of these should use `absolute` instead of `canonicalize`; and
apparently we no longer need to strip the `CANONICAL_CWD` to get tests
passing.
## Summary
This PR addresses an issue on Windows where `std::fs::canonicalize` can
fail or panic when resolving paths on mapped network drives. By
replacing it with `Simplified::simple_canonicalize`, we aim to improve
the robustness and cross-platform compatibility of path resolution.
### Changes
* Updated `CANONICAL_CWD` in `path.rs` to use
`Simplified::simple_canonicalize` instead of `std::fs::canonicalize`.
### Why
* `std::fs::canonicalize` has known issues with resolving paths on
mapped network drives on Windows, which can lead to panics or incorrect
path resolution.
* `Simplified::simple_canonicalize` internally uses
`dunce::canonicalize`, which handles these cases more gracefully,
ensuring better stability and reliability.
## Test Plan
Since `simple_canonicalize` has already been tested in a prior PR, this
change is expected to work without introducing any new issues. No
additional tests are necessary beyond ensuring existing tests pass,
which will confirm the correctness of the change.
## Summary
- The change relates to #6635 is to include compiled python files (.pyc)
in the uv run command.
- After this change `uv run foo.pyc` should spawn `python foo.pyc`.
## Test Plan
- There is a test that uses TestContext to compile and run a simple
python file that prints "Hello World".
- I built the project locally and tried the same with a simple python
file that I had compiled.
## Summary
Closes https://github.com/astral-sh/uv/issues/6840.
## Test Plan
```
❯ ~/workspace/uv/target/debug/uv pip list --verbose
DEBUG uv 0.4.0
DEBUG Searching for Python interpreter in system path
DEBUG Found `cpython-3.12.3-macos-aarch64-none` at `/Users/crmarsh/.local/share/rtx/installs/python/3.12.3/bin/python3` (search path)
DEBUG Using Python 3.12.3 environment at /Users/crmarsh/.local/share/rtx/installs/python/3.12.3/bin/python3
```
As suggested by @samypr100 on #6680:
https://github.com/astral-sh/uv/pull/6680#issuecomment-2313607984
## Summary
Instead of using `UV_INTERNAL__TEST_DIR`, it simply exports `TEMP` when
running Windows jobs.
## Test Plan
I'm going to run this manually under ProcMon on my Windows machine and
see where uv writes temp files, hopefully to the dev drive and not
`%(LOCAL)APPDATA%` or something.
I'm going to commit a dummy code change and look at build time changes
in CI.
## Summary
Closes https://github.com/astral-sh/uv/issues/6699
On cases like the ones described in
https://github.com/astral-sh/uv/issues/6699, `lpReserved2` somehow seems
to report multiple file descriptors that were not tied to any valid
handles. The previous implementation was faulting as it would try to
dereference these invalid handles. This change moves to using `HANDLE`
directly and check if its is_invalid instead before attempting to close
them.
## Test Plan
Manually tested and verified using `busybox-w32` like described in the
issue.
---------
Co-authored-by: konstin <konstin@mailbox.org>
We currently normalize package and extra names and drop the whitespace
from version specifiers, but we were not normalizing the order of the
specifiers. By sorting them we match the behavior of `packaging` and
become independent of build backends reordering specifiers (#6332).
Surprisingly, the snapshot diff isn't large - most people were already
writing sorted specifiers. Still, this will lead to observable
differences in lockfiles between releases in cases where there are
entries in `requires-dist` that were not previously sorted (while the
total number of `requires-dist` is already small compared to the overall
lockfile).
Microsoft Store Pythons do not always register themselves in the
registry, so we port
<58ce131037/PC/launcher2.c (L1744)>
and look them up on the filesystem in known locations.
## Test Plan
So far I've confirmed that we find a store Python when I use `cargo run
python list`, can we make this a part of any of the platform tests
maybe?
Our current strategy of parsing the output of `py --list-paths` to get
the installed python versions on windows is brittle (#6524, missing
`py`, etc.) and it's slow (10ms last time i measured).
Instead, we should behave spec-compliant and read the python versions
from the registry following PEP 514.
It's not fully clear which errors we should ignore and which ones we
need to raise.
We're using the official rust-for-windows crates for accessing the
registry.
Fixes#1521Fixes#6524
Most times we compile with `cargo build`, we don't actually need
`uv-dev`. By making `uv-dev` dependent on a new `dev` feature, it
doesn't get built by default anymore, but only when passing `--features
dev`.
Hopefully a small improvement for compile times or at least system load.
## Summary
We now respect the user-provided upper-bound in for `requires-python`.
So, if the user has `requires-python = "==3.11.*"`, we won't explore
forks that have `python_version >= '3.12'`, for example.
However, we continue to _only_ compare the lower bounds when assessing
whether a dependency is compatible with a given Python range.
Closes https://github.com/astral-sh/uv/issues/6150.
Previously, we were always asking Cargo to rebuild `uv-cli` if
`.git/HEAD` had changed. But in a worktree, `.git` is a file, not a
directory. And the file contains the path to git's internal worktree
state, which also has its own `HEAD` file. So in the case of a worktree,
we read the file and tell Cargo to watch the worktree-specific `HEAD`
file instead of `.git/head`.
The main thing this fixes is that, previously, in a worktree, `cargo
build` would *always* re-compile `uv` even if nothing changed.
This doesn't impact or fix anything in "typical" clones of uv though.
Only in worktrees.
Closes#6196, Closes#6197
## Summary
The interface here is intentionally a bit more limited than `uv pip
compile`, because we don't want `requirements.txt` to be a system of
record -- it's just an export format. So, we don't write annotation
comments (i.e., which dependency is requested from which), we don't
allow writing extras, etc. It's just a flat list of requirements, with
their markers and hashes.
Closes#6007.
Closes#6668.
Closes#6670.
## Summary
resolves https://github.com/astral-sh/uv/issues/6203
## Test Plan
added a test fixing the bug described in the issue.
---------
Co-authored-by: konstin <konstin@mailbox.org>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
- Resolves issue #6690
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
```
$ cargo run python list
```
<!-- How was it tested? -->
---------
Co-authored-by: leaf-soba <leaf-soba@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Whether a package is itself virtual isn't captured in the package
metadata, so we have to compare the sources.
Closes https://github.com/astral-sh/uv/issues/6749.
## Summary
If you're in a directory, and there's workspace above it, we check if
the directory is excluded from the workspace members... But not if it's
_included_ in the first place.
Closes https://github.com/astral-sh/uv/issues/6732.
## Summary
Use a dedicated source type for non-package requirements. Also enables
us to support non-package `path` dependencies _and_ removes the need to
have the member `pyproject.toml` files available when we sync _and_
makes it explicit which dependencies are virtual vs. not (as evidenced
by the snapshot changes). All good things!
## Summary
This PR makes `cargo test | windows` faster in CI.
### Before
### After
## Also
This PR disables the `brotli` feature of `async-compression` since it's
not strictly needed, but this has little to do with the improvements
(it's still less code to build).
This PR introduces additional code in uv tool uninstall to ignore errors
(that only seem to happen on ReFS, ie. on Dev Drives) akin to "the thing
we're trying to delete cannot be deleted because it's already being
deleted".
If `raw_os_error` was stable we could do u32 matching instead of that
`.to_string().contains()` abomination.
## Summary
In theory this problem already existed for `PKG-INFO`, but `egg-info`
would be more common, I think, since it's built in the source tree.
Closes https://github.com/astral-sh/uv/issues/6712.
Changes the `uv init` experience with a focus on working for more
use-cases out of the box.
- Adds `--app` and `--lib` options to control the created project style
- Changes the default from a library with `src/` and a build backend
(`--lib`) to an application that is not packaged (`--app`)
- Hides the `--virtual` option and replaces it with `--package` and
`--no-package`
- `--no-package` is not allowed with `--lib` right now, but it could be
in the future once we understand a use-case
- Creates a runnable project
- Applications have a `hello.py` file which you can run with `uv run
hello.py`
- Packaged applications, e.g., `uv init --app --package` create a
package and script entrypoint, which you can run with `uv run hello`
- Libraries provide a demo API function, e.g., `uv run python -c "import
name; print(name.hello())"` — this is unchanged
Closes#6471
## Summary
We were writing empty lines between the dependencies and the
`tool.uv.sources` table, which led to the `/// script` tag being
unclosed and thus not recognized.
Closes https://github.com/astral-sh/uv/issues/6700.
Should this be user-facing by default? It seems annoying because then
it's unavoidable if you (for whatever reason) have an intentionally
unclosed tag.
Motivated by https://github.com/astral-sh/uv/issues/6700.
## Summary
The basic idea here is: any project can either be a package, or not
("virtual").
If a project is virtual, we don't build or install it.
A project is virtual if either of the following are true:
- `tool.uv.virtual = true` is set.
- `[build-system]` is absent.
The concept of "virtual projects" only applies to workspace member right
now; it doesn't apply to `path` dependencies which are treated like
arbitrary Python source trees.
TODOs that should be resolved prior to merging:
- [ ] Documentation
- [ ] How do we reconcile this with "virtual workspace roots" which are
a little different -- they omit `[project]` entirely and don't even have
a name?
- [x] `uv init --virtual` should create a virtual project rather than a
virtual workspace.
- [x] Running `uv sync` in a virtual project after `uv init --virtual`
shows `Audited 0 packages in 0.01ms`, which is awkward. (See:
https://github.com/astral-sh/uv/pull/6588.)
Closes https://github.com/astral-sh/uv/issues/6511.
## Summary
Tries to improve the following:
```
❯ cargo run sync
Compiling uv-cli v0.0.1 (/Users/crmarsh/workspace/uv/crates/uv-cli)
Compiling uv v0.3.3 (/Users/crmarsh/workspace/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.81s
Running `/Users/crmarsh/workspace/uv/target/debug/uv sync`
Using Python 3.12.1
Creating virtualenv at: .venv
Resolved in 7ms
Audited environment in 0.05ms
```
In this case we don't actually have any dependencies -- should we just
omit `Resolved in...` and perhaps even the audited line?
## Summary
This PR revives https://github.com/astral-sh/uv/pull/4944, which I think
was a good start towards adding `--trusted-host`. Last night, I tried to
add `--trusted-host` with a custom verifier, but we had to vendor a lot
of `reqwest` code and I eventually hit some private APIs. I'm not
confident that I can implement it correctly with that mechanism, and
since this is security, correctness is the priority.
So, instead, we now use two clients and multiplex between them.
Closes https://github.com/astral-sh/uv/issues/1339.
## Test Plan
Created self-signed certificate, and ran `python3 -m http.server --bind
127.0.0.1 4443 --directory . --certfile cert.pem --keyfile key.pem` from
the packse index directory.
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
https://127.0.0.1:8443/simple-html` failed with:
```
error: Request failed after 3 retries
Caused by: error sending request for url (https://127.0.0.1:8443/simple-html/transitive-yanked-and-unyanked-dependency-a-0abad3b6/)
Caused by: client error (Connect)
Caused by: invalid peer certificate: Other(OtherError(CaUsedAsEndEntity))
```
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.1:8443'`
failed with the expected error (invalid resolution) and made valid
requests.
Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.2' -n` also
failed.
As described in #4242, we're currently incorrectly downloading glibc
python-build-standalone on musl target, but we also can't fix this by
using musl python-build-standalone on musl targets since the musl builds
are effectively broken.
We reintroduce the libc detection previously removed in #2381, using it
to detect which libc is the current one before we have a python
interpreter. I changed the strategy a big to support an empty `PATH`
which we use in the tests.
For simplicity, i've decided to just filter out the musl
python-build-standalone archives from the list of available archive,
given this is temporary. This means we show the same error message as if
we don't have a build for the platform. We could also add a dedicated
error message for musl.
Fixes#4242
## Test Plan
Tested manually.
On my ubuntu host, python downloads continue to pass:
```
target/x86_64-unknown-linux-musl/debug/uv python install
```
On alpine, we fail:
```
$ docker run -it --rm -v .:/io alpine /io/target/x86_64-unknown-linux-musl/debug/uv python install
Searching for Python installations
error: No download found for request: cpython-any-linux-x86_64-musl
```
## Summary
We now respect the `environments` field in `uv pip compile --universal`,
e.g.:
```toml
[tool.uv]
environments = ["platform_system == 'Emscripten'"]
```
Closes https://github.com/astral-sh/uv/issues/6641.
## Summary
This is similar to https://github.com/astral-sh/uv/pull/6171 but more
expansive... _Anywhere_ that we test requirements for platform
compatibility, we _need_ to respect the resolver-friendly markers. In
fixing the motivating issue (#6621), I also realized that we had a bunch
of bugs here around `pip install` with `--python-platform` and
`--python-version`, because we always performed our `satisfy` and `Plan`
operations on the interpreter's markers, not the adjusted markers!
Closes https://github.com/astral-sh/uv/issues/6621.
For various reasons, I have a preference for out of tree virtual
environments. Things just work if I symlink, but I don't know that this
is guaranteed, so I thought I'd add a test for it. It looks like there's
another code path that matters (`FoundInterpreter::discover ->
PythonEnvironment::from_root`) for the higher level commands, but
couldn't spot a good place to test that.
Related discussion:
https://github.com/astral-sh/uv/issues/1495#issuecomment-1950442191 /
https://github.com/astral-sh/uv/issues/1578#issuecomment-1949911871
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the behavior a bit of the per-dependency build-isolation
override. That, if the dist name is known, it is passed into the
`SourceBuild::Setup` function. This allows for this override to work for
projects without a `pyproject.toml`, like `detectron2`, using the
specified requirement name. Previously only the `pyproject.toml` name
could be used, which these projects are lacking. An example of a
use-case is given in the *Test Plan* section.
Additionally, the `no_build_isolation_package` has been adding to
`InstallerSettingsRef` and used in `sync` and other commands, as this
was not done yet.
This is useful if you want to **non**-isolate a single package, even
ones without a proper `pyproject.toml`
## Test Plan
<!-- How was it tested? -->
With the following pyproject.toml.
```toml
[project]
name = "detectron-uv"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"detectron2",
"setuptools",
"torch",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.uv.sources]
detectron2 = { git = "https://github.com/facebookresearch/detectron2", rev = "bcfd464d0c810f0442d91a349c0f6df945467143" }
[tool.uv]
no-build-isolation-package = ["detectron2"]
```
The package `detectron2` is now correctly **non**-isolated. Before,
because the logic depended on getting the name from the
`pyproject.toml`, which is lacking in detectron2 you would get the
message, that the source could not be built. This was because it would
still be *isolated* in that case.
With these changes you can now install using (given that you are inside
a workspace with a venv):
```
uv pip install torch setuptools
uv sync
```
This would previously fail with something like:
```
error: Failed to prepare distributions
Caused by: Failed to fetch wheel: detectron2 @ git+https://github.com/facebookresearch/detectron2@bcfd464d0c810f0442d91a349c0f6df945467143
Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:
--- stderr:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/tdejager/Library/Caches/uv/builds-v0/.tmptloDcZ/lib/python3.12/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tdejager/Library/Caches/uv/builds-v0/.tmptloDcZ/lib/python3.12/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
self.run_setup()
File "/Users/tdejager/Library/Caches/uv/builds-v0/.tmptloDcZ/lib/python3.12/site-packages/setuptools/build_meta.py", line 502, in run_setup
super().run_setup(setup_script=setup_script)
File "/Users/tdejager/Library/Caches/uv/builds-v0/.tmptloDcZ/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
exec(code, locals())
File "<string>", line 10, in <module>
ModuleNotFoundError: No module named 'torch'
---
Caused by: This error likely indicates that detectron2 @ git+https://github.com/facebookresearch/detectron2@bcfd464d0c810f0442d91a349c0f6df945467143 depends on torch, but doesn't declare it as a build dependency. If detectron2 @ git+https://github.com/facebookresearch/detectron2@bcfd464d0c810f0442d91a349c0f6df945467143 is a first-party package, consider adding torch to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation`.
```
**Edit**:
Some wording, used isolated where it should be **non**-isolated.
## Summary
Fixes: #6615
Currently, some packages are not installable with `uv`, like `ziglang`
on Linux.
Everything is described in the issue! 😄
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
I added a unit test for the problematic use case.
I also checked that previous unit test are still running in order to
ensure the backward compatibility.
## Summary
This was an oversight. The existing test was (correctly) failing, but
for the wrong reason (failing to build the package during _resolution_).
Updates the snapshot for the deployment from
https://github.com/astral-sh/pypi-proxy/pull/9 — for a while now, we've
only been failing on file requests not registry requests because the
proxy auth was setup wrong.
For users who were using absolute paths in the `pyproject.toml`
previously, this is a behavior change: We now convert all absolute paths
in `path` entries to relative paths. Since i assume that no-one relies
on absolute path in their lockfiles - they are intended to be portable -
I'm tagging this as a bugfix.
Closes https://github.com/astral-sh/uv/pull/6438
Fixes https://github.com/astral-sh/uv/issues/6371
Previously, we excluded these and only looked at system interpreters.
However, it makes sense for this to match the typical Python discovery
experience. We could consider swapping the default... I'm not sure what
makes more sense. If we change the default (as written now) — this could
arguably be a breaking change.
If we don't do this, and `uv run` invokes something like `uv run
--isolated uv pip install foo` uv won't mutate the isolated environment,
it'll mutate whatever outer environment it finds.
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This is a attempt at fixing https://github.com/astral-sh/uv/issues/6486.
It reverts changes made to `pyproject.toml` when sync fails during `uv
add`. This solution felt a little heavy handed and could probably be
improved but it is what happens when locking fails during `uv add` so I
thought it would be a good start.
## Test Plan
<!-- How was it tested? -->
I have added a test case for this to `tests/edit.rs`. It uses
`pytorch==1.0.2` to achieve the desired failure.
## Summary
Indicate the previous version from which uv was upgraded when running
`uv self update`. Thought that it could be useful in some situations to
have a trace of the previous version that was installed.
## Test Plan
Did not find a way to test this, since this heavily relies on being able
to use the installation script and the ability to publish artifacts for
a specific tag.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Two small typo fixes: one in the documentation and one in a comment in
the source code I happened to come across.
## Summary
We accidentally changed the Windows cache directory from
`C:\Users\User\AppData\Local\uv\cache` to
`C:\Users\User\AppData\Local\uv` in v0.3.0. We're considering this a
bug, since it does _not_ match the documentation, and prior to v0.3.0,
we always used the former. This PR migrates back to the previous
location. It should be seamless for users, as we move the cache items to
the new location on startup.
Closes https://github.com/astral-sh/uv/issues/6417.
While working on https://github.com/astral-sh/uv/pull/6389 I discovered
we never checked `cache.get_url` here, which is wrong — though I don't
think it had much effect in practice since the realm would typically
match first. The main problem is that when we call `get_url` later we
hard-code the username to `None` because we assume we checked up here
with the username if present.
## Summary
This reverts commit 7d92915f3d.
I thought this would be a net performance improvement, but we've now had
multiple reports that this made locking _extremely_ slow. I also tested
this today with a very large codebase against a registry that does not
support range requests, and the number of downloads was sort of wild to
watch. Reverting the reduced resolution time by over 50%.
Closes#6104.
## Summary
Mark the new tests requiring Python 3.12.1 specifically as requiring
python-patch feature. This makes the test suite pass again on systems
not having this specific version (and disabling the feature).
## Test Plan
`cargo test` on Gentoo :-).
## Summary
It turns out we weren't applying the collapse logic here, so dev deps
with extras were repeated. This was generally ok... unless we ended up
_dropping_ an extra, in which case, you now have a duplicate.
Closes https://github.com/astral-sh/uv/issues/6380.
In https://github.com/astral-sh/uv/pull/6359, I accidentally made `uv
python install` prefer `.python-version` files over `.python-versions`
files -.-, kind of niche but it's a regression.
Closes https://github.com/astral-sh/uv/issues/6285
Introduces a new problem if the user says `python` but it doesn't exist
with that name:
```
❯ cargo run -q -- -v run python --version
DEBUG uv 0.3.0
DEBUG Found project root: `/Users/zb/workspace/uv`
DEBUG Project `uv` is marked as unmanaged
DEBUG No project found; searching for Python interpreter
DEBUG Reading requests from `/Users/zb/workspace/uv/.python-version`
DEBUG Searching for Python 3.11 in managed installations or system path
DEBUG Found `cpython-3.12.4-macos-aarch64-none` at `/Users/zb/workspace/uv/.venv/bin/python3` (virtual environment)
DEBUG Searching for managed installations at `/Users/zb/Library/Application Support/uv/python`
DEBUG Found `cpython-3.11.9-macos-aarch64-none` at `/opt/homebrew/bin/python3.11` (search path)
DEBUG Using Python 3.11.9 interpreter at: /opt/homebrew/opt/python@3.11/bin/python3.11
DEBUG Running `python --version`
error: Failed to spawn: `python`
Caused by: No such file or directory (os error 2)
```
I'll fix this separately.
## Summary
We're gonna work on a more comprehensive review of whether we should
preserve the username here, but for now, `git@` is effectively a
convention for GitHub and GitLab etc.
Closes https://github.com/astral-sh/uv/issues/6305.
## Test Plan
I guess we don't have infrastructure for testing SSH private keys right
now, but...
```
❯ cargo run init foo
❯ cd foo
❯ cargo run add git+ssh://git@github.com/astral-sh/mkdocs-material-insiders.git
```
For a path dep such as the root project, uv can read metadata statically
from `pyproject.toml` or dynamically from the build backend.
Python's `packaging`
[sorts](cc938f984b/src/packaging/specifiers.py (L777))
specifiers before emitting them, so all build backends built on top of
it - such as hatchling - will change the specifier order compared to
pyproject.toml. The core metadata spec does say "If a field is not
marked as Dynamic, then the value of the field in any wheel built from
the sdist MUST match the value in the sdist", but it doesn't specify if
"match" means string equivalent or semantically equivalent, so it's
arguable if that spec conformant. This change means that the specifiers
have a different ordering when coming from the build backend than when
read statically from pyproject.toml.
Previously, we tried to read path dep metadata in order:
* From the (built wheel) cache (`packaging` order)
* From pyproject.toml (verbatim specifier)
* From a fresh build (`packaging` order)
This behaviour is unstable: On the first run, we cache is cold, so we
read the verbatim specifier from `pyproject.toml`, then we build and
store the metadata in the cache. On the second run, we read the
`packaging` sorted specifier from the cache.
Reproducer:
```shell
rm -rf newproj
uv init -q --no-config newproj
cd newproj/
uv add -q "anyio>=4,<5"
cat uv.lock | grep "requires-dist"
uv sync -q
cat uv.lock | grep "requires-dist"
cd ..
```
```
requires-dist = [{ name = "anyio", specifier = ">=4,<5" }]
requires-dist = [{ name = "anyio", specifier = "<5,>=4" }]
```
A project either has static metadata, so we can read from
pyproject.toml, or it doesn't, and we always read from the build through
`packaging`. We can use this to stabilize the behavior by slightly
switching the order.
* From pyproject.toml (verbatim specifier)
* From the (built wheel) cache (`packaging` order)
* From a fresh build (`packaging` order)
Potentially, we still want to sort the specifiers we get anyway, after
all, the is no guarantee that the specifiers from a build backend are
deterministic. But our metadata reading behavior should be independent
of the cache state, hence changing the order in the PR.
Fixes#6316
## Summary
resolves https://github.com/astral-sh/uv/issues/5915, not entirely sure
if `manylinux_compatible` should be a separate field in the JSON
returned by the interpreter or there's some way to use the existing
`platform` for it.
## Test Plan
ran the below
```
rm -rf .venv
target/debug/uv venv
# commenting out the line below triggers the change..
# target/debug/uv pip install no-manylinux
target/debug/uv pip install cryptography --no-cache
```
is there an easy way to add this into the existing snapshot-based test
suite? looking around to see if there's a way that doesn't involve
something implementation-dependent like mocks.
~update: i think the output does differ between these two, so probably
we can use that.~ i lied - that "building..." output seems to be
discarded.
## Summary
For non-virtual workspaces, these are covered by the _members_. But for
virtual workspaces, they aren't captured anywhere else in the lock. So,
we weren't invalidating `uv.lock` when the dev dependencies changed,
which led to a panic.
Closes https://github.com/astral-sh/uv/issues/6288
## Summary
This ensures that we don't stream output to the `--output-file`, since
other processes may rely on reading it.
Closes https://github.com/astral-sh/uv/issues/6239.
The ADD `MarkerTree` was including the non-deterministic, unstable
`NodeId` in its `Ord` implementation since switching algebraic decision
diagrams. By replacing this with a correct `Ord` implementation, we fix
#6249.
Requires https://github.com/astral-sh/pubgrub/pull/31
This is a fallback mode that we supported when we decided to use PEP 517
builds by default. I can't find a single reference to it on GitHub or in
our issue tracker, so I want to drop support for it as part of v0.3.0.
Executable name requests were being treated as explicit requests to
install into system environments, but I don't think it should be as it's
implicit what environment you'll end up in. Following #4308, we allow
multiple executables to be found so we can filter here.
Concretely, this means `--system` is required to install into a system
environment discovered with e.g. `--python=python`. The flag is still
not required for cases where we're not mutating environment.
These are global and non-specific to the `pip` API, so I think they
should be elevated.
- Ran `UV_CONCURRENT_DOWNLOADS=1 cargo run pip list`; verified that
`downloads` resolved to 1.
- Added `concurrent-downloads = 5` under `[tool.uv]` in
`pyproject.toml`; ran `cargo run pip list`; verified that `downloads`
resolved to 5.
- Ran `UV_CONCURRENT_DOWNLOADS=1 cargo run pip list`; verified that
`downloads` resolved to 1.
This PR moves us to the Linux strategy for our global directories on
macOS. We both feel on the team _and_ have received feedback (in Issues
and Polls) that the `Application Support` directories are more intended
for GUIs, and CLI tools are correct to respect the XDG variables and use
the same directory paths on Linux and macOS.
Namely, we now use:
- `/Users/crmarsh/.local/share/uv/tools` (for tools)
- `/Users/crmarsh/.local/share/uv/python` (for Pythons)
- `/Users/crmarsh/.cache/uv` (for the cache)
The strategy is such that if the `/Users/crmarsh/Library/Application
Support/uv` already exists, we keep using it -- same goes for
`/Users/crmarsh/Library/Caches/uv`, so **it's entirely backwards
compatible**.
If you want to force a migration to the new schema, you can run:
- `uv cache clean`
- `uv tool uninstall --all`
- `uv python uninstall --all`
Which will clean up the macOS-specific directories, paving the way for
the above paths. In other words, once you run those commands, subsequent
`uv` operations will automatically use the `~/.cache` and `~/.local`
variants.
Closes https://github.com/astral-sh/uv/issues/4411.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
- Removes "experimental" labels from command documentation
- Removes preview warnings
- Removes `PreviewMode` from most structs and methods — we could keep it
around but I figure we can propagate it again easily where needed in the
future
- Enables preview behavior by default everywhere, e.g., `uv venv` will
download Python versions
This PR migrates uv's use of `chrono` to `jiff`.
I did most of this work a while back as one of my tests to ensure Jiff
could actually be used in a real world project. I decided to revive
this because I noticed that `reqwest-retry` dropped its Chrono
dependency,
which is I believe the only other thing requiring Chrono in uv.
(Although, we use a fork of `reqwest-middleware` at present, and that
hasn't been updated to latest upstream yet. I wasn't quite sure of the
process we have for that.)
In course of doing this, I actually made two changes to uv:
First is that the lock file now writes an RFC 3339 timestamp for
`exclude-newer`. Previously, we were using Chrono's `Display`
implementation for this which is a non-standard but "human readable"
format. I think the right thing to do here is an RFC 3339 timestamp.
Second is that, in addition to an RFC 3339 timestamp, `--exclude-newer`
used to accept a "UTC date." But this PR changes it to a "local date."
That is, a date in the user's system configured time zone. I think
this makes more sense than a UTC date, but one alternative is to drop
support for a date and just rely on an RFC 3339 timestamp. The main
motivation here is that automatically assuming UTC is often somewhat
confusing, since just writing an unqualified date like `2024-08-19` is
often assumed to be interpreted relative to the writer's "local" time.
e.g.
```
❯ uv pip install httpx==0.26.0 --reinstall-package httpx
Resolved 7 packages in 67ms
Prepared 1 package in 0.95ms
Uninstalled 1 package in 2ms
Installed 1 package in 12ms
~ httpx==0.26.0
```
```
❯ uv add httpx==0.26.0
warning: `uv add` is experimental and may change without warning
warning: `uv.sources` is experimental and may change without warning
Resolved 15 packages in 23ms
Built example @ file:///Users/zb/workspace/example
Prepared 2 packages in 187ms
Uninstalled 2 packages in 2ms
Installed 2 packages in 2ms
~ example==0.1.0 (from file:///Users/zb/workspace/example)
- httpx==0.27.0
+ httpx==0.26.0
```
Motivated by trying to reduce the diff for project updates in `uv add`.
I think it makes sense in general though. We'll also want a special
output for upgrades in the future, e.g. reducing the `httpx` changes in
the second example to a single line indicating the original and
subsequent distribution versions.
I'd like to have
> Reinstalled 1 package in 2ms
but it seems non-trivial to show the timing for it? I think we'd need to
determine what we want to call a "Reinstall" during the operations and
split doing them from the rest of the uninstalls and installs.
## Summary
The strategy here is: if the user provides supported environments, we
use those as the initial forks when resolving. As a result, we never add
or explore branches that are disjoint with the supported environments.
(If the supported environments change, we ignore the lockfile entirely,
so we don't have to worry about any interactions between supported
environments and the preference forks.)
Closes https://github.com/astral-sh/uv/issues/6184.
## Summary
I probably won't land https://github.com/astral-sh/uv/pull/6210 for the
release, but I do want to make one breaking change to prep for it:
renaming `environment-markers` to `resolution-markers` (or
`solution-markers`?) so that it's delineated from the user-defined
markers in that PR.
Indented blocks in Markdown are treated as code blocks, and rustdoc
treats all unadorned code blocks as Rust doctests. Since this wasn't
intended as a doctest and isn't valid Rust, it makes `cargo test --doc`
fail. We fix this by using an explicit code block labeled as `text`.
In https://github.com/astral-sh/uv/issues/5046, we show the tautological
proof:
```
╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20 and only the following versions of jax are available:
jax<=0.4.20
jax==0.4.21
jax==0.4.22
jax==0.4.23
jax==0.4.24
jax==0.4.25
jax==0.4.26
jax==0.4.27
jax==0.4.28
jax==0.4.29
jax==0.4.30
jax==0.4.31
we can conclude that colabfold[alphafold]==1.5.5 depends on jax>=0.4.20.
And because jax>=0.4.20 depends on numpy>=1.26.0, we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0.
(1)
```
This is a part of the error tree because the statement
`colabfold[alphafold]==1.5.5 depends on jax>=0.4.20` is actually a
simplification of `colabfold[alphafold]==1.5.5 depends on
jax>=0.4.20,<0.5.0` and the no versions clause is a proof of that
simplification.
Without simplification, the clause looks like:
```
╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20,<0.5.0 and only the following versions of jax are available:
jax<=0.4.20
jax==0.4.21
jax==0.4.22
jax==0.4.23
jax==0.4.24
jax==0.4.25
jax==0.4.26
jax==0.4.27
jax==0.4.28
jax==0.4.29
jax==0.4.30
jax==0.4.31
we can conclude that colabfold[alphafold]==1.5.5 depends on one of:
jax==0.4.20
jax==0.4.21
jax==0.4.22
jax==0.4.23
jax==0.4.24
jax==0.4.25
jax==0.4.26
jax==0.4.27
jax==0.4.28
jax==0.4.29
jax==0.4.30
jax==0.4.31
And because jax>=0.4.20 depends on numpy>=1.26.0, we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0.
```
I don't think we have a great way to avoid performing the simplification
of the range conditionally and it makes the error simpler to just jump
straight to `colabfold[alphafold]==1.5.5 depends on jax>=0.4.20`.
The derivation for this clause looks like:
```
jax==0.4.20 | ==0.4.21 | ==0.4.22 | ==0.4.23 | ==0.4.24 | ==0.4.25 | ==0.4.26 | ==0.4.27 | ==0.4.28 | ==0.4.29 | ==0.4.30 | ==0.4.31 depends on numpy>=1.26.0
no versions of jax>0.4.20, <0.4.21 | >0.4.21, <0.4.22 | >0.4.22, <0.4.23 | >0.4.23, <0.4.24 | >0.4.24, <0.4.25 | >0.4.25, <0.4.26 | >0.4.26, <0.4.27 | >0.4.27, <0.4.28 | >0.4.28, <0.4.29 | >0.4.29, <0.4.30 | >0.4.30, <0.4.31 | >0.4.31, <0.5.0
colabfold[alphafold]==1.5.5 depends on jax>=0.4.20, <0.5.0
```
So it looks like we can take trees of this form and drop the "no
versions" clause _if_ the ranges are compatible[*]. See [this
comment](https://github.com/astral-sh/uv/pull/6160#discussion_r1720280922)
for a simpler explanation.
With this pull request, the clause simplifies to
```
╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20 and jax>=0.4.20 depends on numpy>=1.26.0,
we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0. (1)
```
Unfortunately, this doesn't change any snapshots in our test suite so
I'm uncertain if the strategy generalizes. In some incorrect iterations
of this logic, the snapshots did reveal my mistakes.
[*] "if the ranges are compatible" includes a bit of hand-waving. I'm
not 100% sure if I've chosen the correct range heuristic here.
## Summary
Passing `--upgrade` to `tool run` is confusing, because it doesn't
upgrade the installed tool. It just causes us to use an isolated tool
environment, which seems wrong.
Closes#3683
Note our semantics do not exactly match the specification so we can
perform algebra on the markers. See the caveats in the documentation
(and in the discussion below).
The test output seems to depend on using Python 3.12.1 specifically.
While I'm not sure how it happens, it seems like these can get out of
sync between CI and local testing. In this case, I had a problem where
the marker expressions emitted locally were tied to Python 3.12.4, but
the tests in CI were tied to Python 3.12.1. Changing the test to require
3.12.1 specifically fixes this.
This initial set is meant to be a basic starting point where we
can test the interaction between 'uv' commands more systematically.
And specifically, with a focus on how the lock file changes.
This adds some variations on 'uv add' and 'uv remove' specifically
for testing changes to the lock file (and not anything else).
We also rejigger 'run_and_format' so that we can use it in other
contexts, particularly for error reporting.
And we add a 'diff_lock' helper for returning the changes made to
a lock file after running a command.
This was only being used in the ecosystem tests. Since we now don't do a
resolve when `uv lock` is run and when the lock file satisfies the
`pyproject.toml`, deterministic checking was removed since it's avoided
by construction. It was removed everywhere else, so we remove it here as
well.
Closes https://github.com/astral-sh/uv/issues/6167
We've been seeing intermittent failures in CI, which we thought were
unexpected HTTP 401s but it actually looks like a panic when handling an
expected HTTP error. I believe the problem is that an early client error
can cause the channel to close and we crash when we unwrap the `send`.
## Summary
Fixes#6177
This ensures a `pyproject.toml` file without a `[project]` table is not
a fatal error for `uv venv`, which is just trying to discover/respect
the project's `python-requires` (#5592).
Similarly, any caught `WorkspaceError` is now also non-fatal and instead
prints a warning message (feeback welcome here, felt less surprising
than e.g. a malformed `pyproject.toml` breaking `uv venv`).
## Test Plan
I added two test cases: `cargo test -p uv --test venv`
Also, existing venv tests were failing for me since I use fish and the
printed activation script was `source .venv/bin/activate.fish` (to
repro, just run the tests with `SHELL=fish`). So added an insta filter
to normalize that.
## Summary
PR #4533 introduced (almost) spec compliant parsing of `.egg-info`
filenames, but added the overly strict requirement that the distribution
version must be present. This causes various `uv pip` operations to fail
in environments where there are `.egg-info` files without a version
component, so loosen this check by making the version component optional
and reading the version from the egg metadata when it is not present.
As an example of the issue, running `uv pip list` on my system currently
results in
```
error: Failed to read metadata from: `/usr/lib/python3.12/site-packages/PySide6.egg-info`
Caused by: The `.egg-info` filename "PySide6.egg-info" is missing a version
```
whereas regular `pip list` succeeds:
```
$ pip list | rg -S pyside
PySide6 6.7.2
```
## Test Plan
This has been tested by altering the `.egg-info` filename tests as
needed and ensuring the full test suite passes locally.
Resolve#6151
## Test Plan
Execution result of `cargo run -- help`
```bash
An extremely fast Python package manager.
Usage: uv [OPTIONS] <COMMAND>
Commands:
run Run a command or script (experimental)
init Create a new project (experimental)
add Add dependencies to the project (experimental)
remove Remove dependencies from the project (experimental)
sync Update the project's environment (experimental)
lock Update the project's lockfile (experimental)
tree Display the project's dependency tree (experimental)
tool Run and install commands provided by Python packages (experimental)
python Manage Python versions and installations (experimental)
pip Manage Python packages with a pip-compatible interface
venv Create a virtual environment
cache Manage uv's cache
version Display uv's version
generate-shell-completion Generate shell completion
help Display documentation for a command
...
```
Execution result of `cargo run -- -h` and `cargo run -- --help`
```bash
An extremely fast Python package manager.
Usage: uv [OPTIONS] <COMMAND>
Commands:
run Run a command or script (experimental)
init Create a new project (experimental)
add Add dependencies to the project (experimental)
remove Remove dependencies from the project (experimental)
sync Update the project's environment (experimental)
lock Update the project's lockfile (experimental)
tree Display the project's dependency tree (experimental)
tool Run and install commands provided by Python packages (experimental)
python Manage Python versions and installations (experimental)
pip Manage Python packages with a pip-compatible interface
venv Create a virtual environment
cache Manage uv's cache
version Display uv's version
help Display documentation for a command
...
```
## Summary
In the resolver, we use release-only semantics to normalize
`python_full_version`. So, if we see `python_full_version < '3.13'`, we
treat that as `(Unbounded, Exclude(3.13))`. `3.13b0` evaluates as `true`
to that range, so we were accepting pre-releases for these markers.
Instead, we need to exclude pre-release segments when performing these
evaluations.
Closes https://github.com/astral-sh/uv/issues/6169.
## Test Plan
Hard to write a test for this because you need a pre-release Python
locally... so:
`echo "sqlalchemy==2.0.32" | cargo run pip compile - --python 3.13 -n`
Resolve#6152
## Summary
## Test Plan
Execution result of `cargo run generate-shell-completion --help`
```bash
Generate shell completion
Usage: uv generate-shell-completion <SHELL>
Arguments:
<SHELL> The shell to generate the completion script for [possible values: bash, elvish, fish, nushell, powershell, zsh]
```
Execution result of `cargo run help generate-shell-completion`
```bash
Generate shell completion
Usage: uv generate-shell-completion <SHELL>
Arguments:
<SHELL>
The shell to generate the completion script for
[possible values: bash, elvish, fish, nushell, powershell, zsh]
```
## Summary
Resolves https://github.com/astral-sh/uv/issues/4537
- First commit avoids overwriting dependencies with different markers.
- Second commit supports adding from requirements files.
## Test Plan
`cargo test`
Now that these incompatibilities are collected into a single range
(https://github.com/astral-sh/uv/pull/6154), we can simplify the range
using the known available versions to reduce verbosity.
There were different `PubGrubPackage` types so they never matched the
available versions set! Luckily, the available versions are agnostic to
the markers and optional dependencies so we can just broaden to using
`PackageName` as a lookup key.
Addresses yet another complaint in
https://github.com/astral-sh/uv/issues/5046
I need this for debugging error messages.
I used an environment variable instead of a trace log so you can do
`UV_INTERNAL__SHOW_DERIVATION_TREE=1` and run a test to see the tree in
the test snapshot without further changes.
e.g.
```rust
// Resolving should fail.
uv_snapshot!(context.filters(), context.lock().arg("--preview").current_dir(&workspace), @r###"
success: false
exit_code: 1
----- stdout -----
UV_INTERNAL__SHOW_DERIVATION_TREE
root==0a0.dev0 depends on foo*
root==0a0.dev0 depends on bar[some-extra]*
foo==0.1.0 depends on anyio==4.1.0
bar[some-extra]==0.1.0 depends on anyio==4.2.0
no versions of bar[some-extra]<0.1.0 | >0.1.0
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
× No solution found when resolving dependencies:
╰─▶ Because only bar[some-extra]==0.1.0 is available and bar[some-extra] depends on anyio==4.2.0, we can conclude that all versions of bar[some-extra] depend on anyio==4.2.0.
And because foo depends on anyio==4.1.0, we can conclude that foo and all versions of bar[some-extra] are incompatible.
And because your workspace requires bar[some-extra] and foo, we can conclude that your workspace's requirements are unsatisfiable.
"###
);
```
We have bad error messages for optional (extra) dependencies and
development dependencies in workspaces:
1. We weren't showing the full package, so we'd drop `:dev` and
`[extra]` by accident
2. We didn't include derived packages, e.g., `member[extra]` in tree
processing collapse operation, so we'd include extra clauses like the
ones we removed in #6092
Also
- Reverts
f0de4f71f2
— it turns out it wasn't quite correct and it didn't seem worth using
the custom incompatibility anymore.
- Fixes a bug in the display of `package:dev` which was not showing
`:dev` for some variants (see 94d8020b58)
## Summary
Normalize all `python_version` markers to their equivalent
`python_full_version` form. This avoids false positives in forking
because we currently cannot detect any relationships between the two
forms. It also avoids subtle bugs due to the truncating semantics of
`python_version`. For example, given `requires-python = ">3.12"`, we
currently simplify the marker `python_version <= 3.12` to `false`.
However, the version `3.12.1` will be truncated to `3.12` for
`python_version` comparisons, and thus it satisfies the python
requirement and evaluates to `true`.
It is possible to simplify back to `python_version` when writing markers
to the lockfile. However, the equivalent `python_full_version` markers
are often clearer and easier to simplify, so I lean towards leaving them
as `python_full_version`.
There are *a lot* of snapshot updates from this change. I'd like more
eyes on the transformation logic in `python_version_to_full_version` to
ensure that they are all correct.
Resolves https://github.com/astral-sh/uv/issues/6125.
While it's slightly more convenient to log this where we were, it was
pretty unhelpful e.g.
```
DEBUG Interpreter meets the requested Python: `Python >=3.9`
```
What interpreter are we referring to here?
Includes the changes from https://github.com/astral-sh/uv/pull/6071 but
takes them way further.
When we have the set of available versions for a package, we can do a
much better job displaying an error.
For example:
```
❯ uv add 'httpx>999,<9999'
× No solution found when resolving dependencies:
╰─▶ Because only the following versions of httpx are available:
httpx<=999
httpx>=9999
and example==0.1.0 depends on httpx>999,<9999, we can conclude that example==0.1.0 cannot be used.
And because only example==0.1.0 is available and you require example, we can conclude that the requirements are unsatisfiable.
```
The resolver has demonstrated that the requested range cannot be used
because there are only versions in ranges _outside_ the requested range.
However, the display of the range of available versions is pretty bad!
We say there are versions of httpx available in ranges that definitely
have no versions available.
With this pull request, the error becomes:
```
❯ uv add 'httpx>999,<9999'
× No solution found when resolving dependencies:
╰─▶ Because only httpx<=1.0.0b0 is available and example depends on httpx>999,<9999, we can conclude that example's
requirements are unsatisfiable.
And because your workspace requires example, we can conclude that your workspace's requirements are unsatisfiable.
```
We achieve this by:
1. Dropping ranges disjoint with the range of available versions, e.g.,
this removes `httpx>=9999`
2. Replacing ranges that capture the _entire_ range of available
versions with the smaller range, e.g., this replaces `httpx<=999` with
`<=1.0.0b0`.
~Note that when we perform (2), we may include an additional bound that
is not relevant, e.g., we include the lower bound of `>=0.6.7`. This is
a bit extraneous, but I don't think it's confusing. We can consider some
advanced logic to avoid that later.~ (edit: I did this, it wasn't hard)
We also improve error messages when there is _only_ one version
available by showing that version instead of a range.
## Summary
Gives the caller control over how messages are reported back to the
user. Also merges the index-location validation into the lock, since
we're already iterating over the packages.
## Summary
This is no longer required since we no longer implement `Eq` on `Lock`.
It will also sometimes be "wrong" as of #6076, since we now apply
different `requires-python` filtering to different parts of the tree
during resolution.
## Summary
Using https://github.com/astral-sh/uv/issues/6064 as a motivating
example: at present, on main, we're not properly propagating the
`Requires-Python` simplifications. In that case, for example, we end up
solving for a branch with `python_version < 3.11`, and a branch `>=
3.11`, even though `Requires-Python` is `>=3.11`. Later, when we get to
the graph, we apply version simplification based on `Requires-Python`,
which causes us to _remove_ the `python_version < 3.11` markers
entirely, leaving us with duplicate dependencies for `pylint`.
This PR instead tries to ensure that we always apply this narrowing to
requirements and forks, so that we don't need to apply the same
simplification when constructing the graph at all.
Closes https://github.com/astral-sh/uv/issues/6064.
Closes#6059.
In particular, I added this as a hack to avoid a kinda of
instability that was caused by our marker code not correctly
detecting markers that were always false. But that has since
been fixed.
Removing this code doesn't change any tests. Arguably it
should be possible to come up with a test that failed with
this hack inserted but succeeded without it. In particular,
with this hack, new forks were being prevented from being
added even when they ought to be added, e.g., when preferences
get updated.
## Summary
This PR changes the definition of `--locked` from:
> Produces the same `Lock`
To:
> Passes `Lock::satisfies`
This is a subtle but important difference. Previous, if
`Lock::satisfies` failed, we would run a resolution, then do
`existing_lock == lock`. If the two weren't equal, and `--locked` was
specified, we'd throw an error.
The equality check is hard to get right. For example, it means that we
can't ship #6076 without changing our marker representation, since the
deserialized lockfile "loses" some of the internal marker state that
gets accumulated during resolution.
The downside of this change is that there could be scenarios in which
`uv lock --locked` fails even though the lockfile would actually work
and the exact TOML would be unchanged. But... I think it's ok if
`--locked` fails after the user modifies something?
Extends https://github.com/astral-sh/uv/pull/6092 to improve resolver
error messages for workspaces that have a single member.
As before, this requires a two-step approach of
1. Traversing the derivation tree and collapsing some members. In this
case, we drop the empty root node in favor of the project.
2. Using special-case formatting for packages. In this case, the
workspace package is referred to with "your project" instead of its
name.
An extension of #6090 that replaces #6066.
In brief,
1. Workspace member names are passed to the resolver for no solution
errors
2. There is a new derivation tree pre-processing step that trims
`NoVersion` incompatibilities for workspace members from the derivation
tree. This avoids showing redundant clauses like `Because only
bird==0.1.0 is available and bird==0.1.0 depends on anyio==4.3.0, we can
conclude that all versions of bird depend on anyio==4.3.0.`. As a minor
note, we use a custom incompatibility kind to mark these
incompatibilities at resolution-time instead of afterwards.
3. Root dependencies on workspace members say `your workspace requires
bird` rather than `you require bird`
4. Workspace member package display omits the version, e.g., `bird`
instead of `bird==0.1.0`
5. Instead of reporting a workspace member as unusable we note that its
requirements cannot be solved, e.g., `bird's requirements are
unsatisfiable` instead of `bird cannot be used`.
6. Instead of saying `your requirements are unsatisfiable` we say `your
workspace's requirements are unsatisfiable` when in a workspace, since
we're not in a "provide direct requirements" paradigm.
As an annoying but minor implementation detail, `PackageRange` now
requires access to the `PubGrubReportFormatter` so it can determine if
it is formatting a workspace member or not. We could probably improve
the abstractions in the future.
As a follow-up, we should additional special casing for "single project"
workspaces to avoid mention of the workspace concept in simple projects.
However, it looks like this will require additional tree manipulations
so I'm going to keep it separate.
## Summary
Historically, in order to "resolve from a lockfile", we've taken the
lockfile, used it to pre-populate the in-memory metadata index, then run
a resolution. If the resolution didn't match our existing resolution, we
re-resolved from scratch.
This was an appealing approach because (in theory) it didn't require any
dedicated logic beyond pre-populating the index. However, it's proven to
be _really_ hard to get right, because it's a stricter requirement than
we need. We just need the current lockfile to _satisfy_ the requirements
provided by the user. We don't actually need a second resolution to
produce the exact same result. And it's not uncommon that this second
resolution differs, because we seed it with preferences, which
fundamentally changes its course. We've worked hard to minimize those
"instabilities", but they're still present.
The approach here is intended to be much simpler. Instead of resolving
from the lockfile, we just check if the current resolution satisfies the
state of the workspace. Specifically, we check if the lockfile (1)
contains all the relevant members, and (2) matches the metadata for all
dependencies, recursively. (We skip registry dependencies, assuming that
they're immutable.)
This may actually be too conservative, since we can have resolutions
that satisfy the requirements, even if the requirements have changed
slightly. But we want to bias towards correctness for now.
My hope is that this scheme will be more performant, simpler, and more
robust.
Closes https://github.com/astral-sh/uv/issues/6063.
## Summary
This was added in https://github.com/astral-sh/uv/pull/5405 but is now
the cause of an instability in `github_wikidata_bot`. Specifically, on
the initial run, we fork in `pydantic==2.8.2`, via:
```
Requires-Dist: typing-extensions>=4.12.2; python_version >= '3.13'
Requires-Dist: typing-extensions>=4.6.1; python_version < '3.13'
```
In the end, we resolve a single version of `typing-extensions`
(`4.12.2`)... But we don't recognize the two resolutions as the "same
graph", because we propagate the fork markers, and so the "edges" have
different markers on them...
In the second run through, when we have the forks in advance, we don't
split on Pydantic... We just try to solve from the root with the current
forks. This is fundamentally different and I fear it will be the cause
of many instabilities. But removing this graph check fixes the proximate
issue.
I don't really understand why this was added since there was no test
coverage in the PR.
## Summary
When constructing the `Resolution`, we only propagated the fork markers
to the package node, but not the extras node. This led to cases in which
an extra could be included unconditionally or otherwise diverge from the
base package version.
Closes https://github.com/astral-sh/uv/issues/6062.
## Summary
Right now, we store the environment markers in a `BTreeSet` -- so
they're sorted, but the sort doesn't really tell us anything. I think we
should instead store them in the order in which we solved. I thought
this might fix an instability (it didn't), but I think it's still good
to ensure we solve in the same order.
I also changed from `Option<Vec>` to just `Vec`, since there was no
distinction between `None` and empty.
## Summary
We retain them if you use `--raw-sources`, but otherwise they're
removed. We still respect them in the subsequent `uv.lock` via an
in-process store.
Closes#6056.
## Summary
Now, if you resolve against a registry, then swap it out for another, we
won't reuse the lockfile. (If you don't provide any registry
configuration, then we won't enforce this, so that `uv lock --index-url
foo` and `uv lock` is stable.)
Closes https://github.com/astral-sh/uv/issues/5920.
This example came up in discussion and it was initially unclear whether
we should try to support it. Specifically, by automatically assuming
that the `datasets < 2.19` dependency had a marker corresponding to the
negation of the conjunction of the other sibling markers for that same
package. But this was deemed, I think, a little too magical.
This in turn implies that whenever there are sibling dependencies with
overlapping marker expressions, their version constraints also need to
be overlapping. Otherwise, for any marker environment that matches both
marker expressions, it would be impossible to select a single version.
The test in this case has this comment:
```
/// If a dependency requests a prerelease version with an overlapping marker expression,
/// we should prefer the prerelease version in both forks.
```
With this setup:
```
let pyproject_toml = context.temp_dir.child("pyproject.toml");
pyproject_toml.write_str(indoc! {r#"
[project]
name = "example"
version = "0.0.0"
dependencies = [
"cffi >= 1.17.0rc1 ; os_name == 'Linux'"
]
requires-python = ">=3.11"
"#})?;
let requirements_in = context.temp_dir.child("requirements.in");
requirements_in.write_str(indoc! {"
cffi
.
"})?;
```
The change in this commit _seems_ more correct that what we had,
although it does seem to contradict the comment. Namely, in the `os_name
!= "Linux"` fork, we don't prefer the pre-release version since the
`cffi >= 1.17.0rc1` bound doesn't apply.
It's not quite clear what to do in this instance.
I believe these are all changes that aren't necessarily
expected, but also seem harmless. Like the order in which
fork markers are written to the lock file. (Although one
wonders if we should fix that once and for all by defining
a complete sort function for forks.)
At a high level, this PR adds a smattering of new tests that
effectively snapshot the output of `uv lock` for a selection of
"ecosystem" projects. That is, real Python projects for which we expect
`uv` to work well with.
The main idea with these tests is to get a better idea of how changes
in `uv` impact the lock files of real world projects. For example,
we're hoping that these tests will help give us data for how #5733
differs from #5887.
This has already revealed some bugs. Namely, re-running `uv lock` for a
second time will produce a different lock file for some projects. So to
prioritize getting the tests added, for those projects, we don't do the
deterministic checking.
## Summary
Our current handling of `--find-links` merges the entries in each index.
As a result, we can end up with `AnnotatedDist` entries that reference
distributions across indexes.
I'd like to change `--find-links` such that each `--find-links` entry is
just treated as its own index (so, e.g., if `requests` exists in the
first `--find-links` entry, we don't even check the registry by
default), which would _also_ fix this problem automatically. But that's
a behavior change... So for now, in the lockfile, we filter
distributions that don't match the source index URL.
There are two cases to consider:
- There's a source distribution. Then, for the ID to reference the
`--find-links` registry, the source distribution _must_ have come from
the `--find-links` entry, so it's fine to discard any wheels from the
"wrong" registry without breaking any compatibility guarantees.
- There's no source distribution. Then the best wheel must come from the
`--find-links` registry. We might lose some platform coverage by
discarding the other wheels, but it shouldn't break any of the
"guarantees", since we have at least one wheel that fits in the version
range.
Closes https://github.com/astral-sh/uv/issues/6015.
## Summary
Added the actual error message to the warning when uv fails to parse
`pyproject.toml`.
Resolves https://github.com/astral-sh/uv/issues/5934
## Test Plan
Took the case from the issue:
- have `pyproject.toml` which contains
```
[tool.uv]
foobar = false
```
-
```
$ uv venv --preview -v
```
- Expect the message that contains the actual problem in the
`pyproject.toml` like:
```
warning: Failed to parse `pyproject.toml` during settings discovery: unknown field `foobar`; skipping...
```
## Summary
A lot of the existing tests were no-ops. For convenience, we now use the
trick of: install from Test PyPI (to get an outdated "latest"), then
upgrade from PyPI.
Right now, the URL gets out-of-sync with the install path, since the
install path is canonicalized. This leads to a subtle error on Windows
(in CI) in which we don't preserve caching across resolution and
installation.
Surprisingly, this is a lockfile schema change: We can't store relative
paths in urls, so we have to store a `filename` entry instead of the
whole url.
Fixes#4355
## Summary
We now persist the `ResolverInstallerOptions` when writing out a tool
receipt. When upgrading, we grab the saved options, and merge with the
command-line arguments and user-level filesystem settings (CLI > receipt
> filesystem).
The loose consensus is that "fetch" doesn't have much meaning and that a
boolean flag makes more sense from the command line.
1. Adds `--allow-python-downloads` (hidden, default) and
`--no-python-downloads` to the CLI to quickly enable or disable
downloads
2. Deprecates `--python-fetch` in favor of the options from (1)
3. Removes `python-fetch` in favor of a `python-downloads` setting
5. Adds a `never` variant to the enum, allowing even explicit installs
to be disabled via the configuration file
## Test plan
I tested this with various `pyproject.toml`-level settings and `uv venv
--preview --python 3.12.2` and `uv python install 3.12.2` with and
without the new CLI flags.
Warn when there are missing bounds on transitive dependencies with
`--resolution lowest`.
Implemented as a lazy resolution graph check. Dev deps are odd because
they are missing the edge from the root that extras have (they are
currently orphans in the resolution graph), but this is more complex to
solve properly because we can put dev dep information in a `Requirement`
so i special cased them here.
Closes#2797
Should help with #1718
---------
Co-authored-by: Ibraheem Ahmed <ibraheem@ibraheem.ca>
## Summary
This PR rewrites the `MarkerTree` type to use algebraic decision
diagrams (ADD). This has many benefits:
- The diagram is canonical for a given marker function. It is impossible
to create two functionally equivalent marker trees that don't refer to
the same underlying ADD. This also means that any trivially true or
unsatisfiable markers are represented by the same constants.
- The diagram can handle complex operations (conjunction/disjunction) in
polynomial time, as well as constant-time negation.
- The diagram can be converted to a simplified DNF form for user-facing
output.
The new representation gives us a lot more confidence in our marker
operations and simplification, which is proving to be very important
(see https://github.com/astral-sh/uv/pull/5733 and
https://github.com/astral-sh/uv/pull/5163).
Unfortunately, it is not easy to split this PR into multiple commits
because it is a large rewrite of the `marker` module. I'd suggest
reading through the `marker/algebra.rs`, `marker/simplify.rs`, and
`marker/tree.rs` files for the new implementation, as well as the
updated snapshots to verify how the new simplification rules work in
practice. However, a few other things were changed:
- [We now use release-only comparisons for `python_full_version`, where
we previously only did for
`python_version`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/algebra.rs#L522).
I'm unsure how marker operations should work in the presence of
pre-release versions if we decide that this is incorrect.
- [Meaningless marker expressions are now
ignored](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/parse.rs#L502).
This means that a marker such as `'x' == 'x'` will always evaluate to
`true` (as if the expression did not exist), whereas we previously
treated this as always `false`. It's negation however, remains `false`.
- [Unsatisfiable markers are written as `python_version <
'0'`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/tree.rs#L1329).
- The `PubGrubSpecifier` type has been moved to the new `uv-pubgrub`
crate, shared by `pep508-rs` and `uv-resolver`. `pep508-rs` also depends
on the `pubgrub` crate for the `Range` type, we probably want to move
`pubgrub::Range` into a separate crate to break this, but I don't think
that should block this PR (cc @konstin).
There is still some remaining work here that I decided to leave for now
for the sake of unblocking some of the related work on the resolver.
- We still use `Option<MarkerTree>` throughout uv, which is unnecessary
now that `MarkerTree::TRUE` is canonical.
- The `MarkerTree` type is now interned globally and can potentially
implement `Copy`. However, it's unclear if we want to add more
information to marker trees that would make it `!Copy`. For example, we
may wish to attach extra and requires-python environment information to
avoid simplifying after construction.
- We don't currently combine `python_full_version` and `python_version`
markers.
- I also have not spent too much time investigating performance and
there is probably some low-hanging fruit. Many of the test cases I did
run actually saw large performance improvements due to the markers being
simplified internally, reducing the stress on the old `normalize`
routine, especially for the extremely large markers seen in
`transformers` and other projects.
Resolves https://github.com/astral-sh/uv/issues/5660,
https://github.com/astral-sh/uv/issues/5179.
## Summary
This PR adds a `DistExtension` field to some of our distribution types,
which requires that we validate that the file type is known and
supported when parsing (rather than when attempting to unzip). It
removes a bunch of extension parsing from the code too, in favor of
doing it once upfront.
Closes https://github.com/astral-sh/uv/issues/5858.
## Summary
I think this seems reasonable... Otherwise, we might not go back to PyPI
to revalidate the list of available versions despite the user passing
`--upgrade`.
## Summary
Previously, we wouldn't respect configuration files in directories
_above_ a workspace root. But this is somewhat problematic, because any
`pyproject.toml` will define a workspace root...
Instead, I think we should _start_ the search at the workspace root, but
go above it if necessary.
Closes: #5929.
See: https://github.com/astral-sh/uv/pull/4295.
## Summary
Resolves#5188. Most of the changes involve creating a new function in
`tool/common.rs` to contain the common functionality previously found in
`tool/install.rs`.
## Test Plan
`cargo test`
```console
❯ ./target/debug/uv tool upgrade black
warning: `uv tool upgrade` is experimental and may change without warning.
Resolved 6 packages in 25ms
Uninstalled 1 package in 3ms
Installed 1 package in 19ms
- black==23.1.0
+ black==24.4.2
Installed 2 executables: black, blackd
```
e.g.
```
❯ cargo run -- venv --no-system
Blocking waiting for file lock on build directory
Compiling uv v0.2.34 (/Users/zb/workspace/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.85s
Running `target/debug/uv venv --no-system`
warning: The `--no-system` flag has no effect, a system Python interpreter is always used in `uv venv`
Using Python 3.12.4 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ cargo run -- venv --system
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
Running `target/debug/uv venv --system`
warning: The `--system` flag has no effect, a system Python interpreter is always used in `uv venv`
Using Python 3.12.4 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
## Summary
This _used_ to be true but we now require fetching metadata for all
distributions even with `--no-deps` since, e.g., we validate that any
declared extras exist.
## Summary
Initially, we showed _all_ resolver and installer output in `uv run` and
`uv tool run`, since it was way too much for workhorse commands. Then,
we moved to showing _no_ output by default, which was way too little --
you had no idea why anything was happening, and commands appeared to
hang.
This PR adds a more nuanced middle-ground. With `--verbose`, we continue
to show everything. But by default, in `uv run` and `uv tool run`...
- During resolution, we show any "Building" and "Build" messages, if you
need to build a source distribution. But we don't show any other output.
(This _could_ be too little for expensive resolutions; we may want to
show a spinner.)
- If there are no changes to be made after resolving, we don't show any
other output.
- If we have to install, we show the progress bars for downloads (which
disappear on completion) followed by a single summary line stating the
number of packages installed.
This feels pretty good, in my limited testing. When everything is built
/ cached, you don't get _any_ additional output. When there's work to
do, you have a sense for what's happening, and we leave you with a
single summary line ("Installed X packages") at the end.
Closes https://github.com/astral-sh/uv/issues/5758.
## Test Plan
Notice that the first `tool run` ends with an install line; the second
shows no additional output:
If you run `uv run` in a package for the first time, we _do_ tell you
that we're building / built it:
But on the second run, there's no output:
If you add a `--with`, we'll show you all the installer progress bars
(which disappear once they're done), and then a single summary line:
Currently, the entry for a package+version+source table is called
`distribution`. That is incorrect, the `sdist` and `wheel` fields inside
of that table are distributions, the table itself is for a package. We
also align ourselves closer with PEP 751.
I went through `lock.rs` and renamed all occurrences of "distribution"
that actually referred to a "package".
This change invalidates all existing lockfiles.
Bikeshedding: Do we call it `package` or `packages`? See also
https://github.com/python/peps/pull/3877
`package` is nice because it looks like a header:
```toml
[[package]]
name = "anyio"
version = "4.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "idna" },
{ name = "sniffio" },
]
sdist = { url = "3970183622d484d08e3285104333d3/anyio-4.3.0.tar.gz", hash = "sha256:f75253795a87df48568485fd18cdd2a3fa5c4f7c5be8e5e36637733fce06fed6", size = 159642 }
wheels = [
{ url = "2f20c40b45242c0b33774da0e2e34f/anyio-4.3.0-py3-none-any.whl", hash = "sha256:048e05d0f6caeed70d731f3db756d35dcc1f35747c8c403364a8332c630441b8", size = 85584 },
]
```
`packages` is nice because the field is not a single entry, but a list.
2/3 for https://github.com/astral-sh/uv/issues/4893
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Whenever we call `resolve`, we immediately call `fetch` after. And in
some cases `resolve` actually calls `fetch` internally. It seems a lot
simpler to just merge these into one method that returns a `Fetch`
(which itself contains the fully-resolved URL).
Closes https://github.com/astral-sh/uv/issues/5876.
There are three options that determine resolver behavior:
* resolution mode
* prerelease mode
* exclude newer
They are different from the other top level options: If they mismatch,
we recreate the resolution. To distinguish them from the rest of the
lockfile, we group them under an `[options]` header.
1/3 for #4893
Following #5869, the documentation has some less-than-helpful
suggestions to use `uv help python` for details — we should link to the
`uv python` section instead.
## Summary
We were dropping the query and fragment in the wrong place, so the URLs
didn't match up after resolving from an existing lockfile.
Closes https://github.com/astral-sh/uv/issues/5851.
## Summary
Very subtle bug. The scenario is as follows:
- We resolve: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git" }`
- The user then changes the request to: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git", rev =
"44d2f4b19d6837ea990c16f494bdf7543d57483d" }`
- When we go to re-lock, we note two facts:
1. The "default branch" resolves to
`44d2f4b19d6837ea990c16f494bdf7543d57483d`.
2. The metadata for `44d2f4b19d6837ea990c16f494bdf7543d57483d` is
(whatever we grab from the lockfile).
- In the resolver, we then ask for the metadata for
`44d2f4b19d6837ea990c16f494bdf7543d57483d`. It's already in the cache,
so we return it; thus, we never add the
`44d2f4b19d6837ea990c16f494bdf7543d57483d` ->
`44d2f4b19d6837ea990c16f494bdf7543d57483d` mapping to the Git resolver,
because we never have to resolve it.
This would apply for any case in which a requested tag or branch was
replaced by its precise SHA. Replacing with a different commit is fine.
It only applied to `tool.uv.sources`, and not PEP 508 URLs, because the
underlying issue is that we aren't consistent about "automatically"
extracting the precise commit from a Git reference.
Closes https://github.com/astral-sh/uv/issues/5860.
## Summary
This is an experimental PR to replace more unsafe calls with more rust
while still trying to keep the binary size small enough. These changes
roughly increase the size of the trampolines to about 40kb~. This is a
alternate PR to https://github.com/astral-sh/uv/pull/5751.
The primary changes here include
* Switch to use rust path components for ease of path management
* Leverage `std::process::exit` for process exit and cleanup
* Use `std::io::Error::last_os_error` for IO Errors to remove
`FormatMessage` complexity
* Use `std::env::current_exe` to get the current executable instead of
`GetModuleFileNameA`
## Test Plan
Added one more existing test case to trampoline tests.
Still need to verify dunce::canonicalize is desired or not on
find_python_exe.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
We need to avoid using incompatible versions for build dependencies that
are also part of the resolved
environment. This is a very subtle issue, but: when locking, we don't
enforce platform
compatibility. So, if we reuse the resolver state to install, and the
install itself has to
preform a resolution (e.g., for the build dependencies of a source
distribution), that
resolution may choose incompatible versions.
The key property here is that there's a shared package between the build
dependencies and the
project dependencies.
Closes https://github.com/astral-sh/uv/issues/5836.
This already rejects `pyproject.toml`... but because the schema
validation is relaxed (we allow unknown fields, and all fields are
optional), a `pyproject.toml` doesn't get properly rejected here.
This PR makes the schema stricter, but in a safe way (by adding the
other `tool.uv` fields, like `workspace`, as any).
Closes#5832.
## Summary
We allow the use of (e.g.) `.whl.metadata` files when `--no-binary` is
enabled, so it makes sense that we'd also also allow wheels to be
downloaded for metadata extraction. So now, we validate `--no-binary` at
install time, rather than metadata-fetch time.
Closes https://github.com/astral-sh/uv/issues/5699.
## Summary
This fixes a bug introduced by
https://github.com/astral-sh/uv/pull/5232. It turns out that the
`universal_disjoint_base_or_local_requirement` test does not actually do
what it was meant to because of the incorrect python requirement. With a
valid python requirement, it fails on `main`. The problem is that we try
to exclude the original base version from the range of allowed versions
to try and prefer local versions. However, in the test, there is a
branch that depends on the non-local version, with no applicable local
in its fork. We should remove this exclusion as prioritization is
handled by the candidate resolver.
I don't think this will save any time in serialization, but it should
save us some deserialization, since we only need to parse URLs for the
packages we use...
## Summary
Okay, I tested this against...
- Our public "private" proxy
- Fury
- AWS CodeArtifact
- Azure Artifacts
It took a long time.
All of them work as expected with this approach: we omit the credentials
from the lockfile, then wire them back up when the index URL is provided
during subsequent operations.
Closes https://github.com/astral-sh/uv/issues/5119.
Part of #4454
e.g.
```
$ uv add --help
Add one or more packages to the project requirements
Usage: uv add [OPTIONS] <REQUIREMENTS>...
Arguments:
<REQUIREMENTS>... The packages to add, as PEP 508 requirements (e.g., `ruff==0.5.0`)
Options:
--dev Add the requirements as development dependencies
--optional <OPTIONAL> Add the requirements to the specified optional dependency group
--no-editable Don't add the requirements as editables
--raw-sources Add source requirements to `project.dependencies`, rather than `tool.uv.sources`
--rev <REV> Specific commit to use when adding from Git
--tag <TAG> Tag to use when adding from git
--branch <BRANCH> Branch to use when adding from git
--extra <EXTRA> Extras to activate for the dependency; may be provided more than once
--locked Assert that the `uv.lock` will remain unchanged
--frozen Add the requirements without updating the `uv.lock` file
--package <PACKAGE> Add the dependency to a specific package in the workspace
-p, --python <PYTHON> The Python interpreter into which packages should be installed. [env: UV_PYTHON=]
Index options:
-i, --index-url <INDEX_URL> The URL of the Python package index (by default: <https://pypi.org/simple>) [env: UV_INDEX_URL=]
--extra-index-url <EXTRA_INDEX_URL> Extra URLs of package indexes to use, in addition to `--index-url` [env: UV_EXTRA_INDEX_URL=]
-f, --find-links <FIND_LINKS> Locations to search for candidate distributions, in addition to those found in the registry indexes
--no-index Ignore the registry index (e.g., PyPI), instead relying on direct URL dependencies and those provided via `--find-links`
--index-strategy <INDEX_STRATEGY> The strategy to use when resolving against multiple index URLs [env: UV_INDEX_STRATEGY=] [possible values: first-index, unsafe-first-match, unsafe-best-match]
--keyring-provider <KEYRING_PROVIDER> Attempt to use `keyring` for authentication for index URLs [env: UV_KEYRING_PROVIDER=] [possible values: disabled, subprocess]
Resolver options:
-U, --upgrade Allow package upgrades, ignoring pinned versions in any existing output file
-P, --upgrade-package <UPGRADE_PACKAGE> Allow upgrades for a specific package, ignoring pinned versions in any existing output file
--resolution <RESOLUTION> The strategy to use when selecting between the different compatible versions for a given package requirement [env: UV_RESOLUTION=] [possible values: highest, lowest, lowest-direct]
--prerelease <PRERELEASE> The strategy to use when considering pre-release versions [env: UV_PRERELEASE=] [possible values: disallow, allow, if-necessary, explicit, if-necessary-or-explicit]
--exclude-newer <EXCLUDE_NEWER> Limit candidate packages to those that were uploaded prior to the given date [env: UV_EXCLUDE_NEWER=]
Installer options:
--reinstall Reinstall all packages, regardless of whether they're already installed. Implies `--refresh`
--reinstall-package <REINSTALL_PACKAGE> Reinstall a specific package, regardless of whether it's already installed. Implies `--refresh-package`
--link-mode <LINK_MODE> The method to use when installing packages from the global cache [env: UV_LINK_MODE=] [possible values: clone, copy, hardlink, symlink]
--compile-bytecode Compile Python files to bytecode after installation
Build options:
-C, --config-setting <CONFIG_SETTING> Settings to pass to the PEP 517 build backend, specified as `KEY=VALUE` pairs
--no-build Don't build source distributions
--no-build-package <NO_BUILD_PACKAGE> Don't build source distributions for a specific package
--no-binary Don't install pre-built wheels
--no-binary-package <NO_BINARY_PACKAGE> Don't install pre-built wheels for a specific package
Cache options:
-n, --no-cache Avoid reading from or writing to the cache, instead using a temporary directory for the duration of the operation [env: UV_NO_CACHE=]
--cache-dir <CACHE_DIR> Path to the cache directory [env: UV_CACHE_DIR=]
--refresh Refresh all cached data
--refresh-package <REFRESH_PACKAGE> Refresh cached data for a specific package
Python options:
--python-preference <PYTHON_PREFERENCE> Whether to prefer using Python installations that are already present on the system, or those that are downloaded and installed by uv [possible values: only-managed, managed, system, only-system]
--python-fetch <PYTHON_FETCH> Whether to automatically download Python when required [possible values: automatic, manual]
Global options:
-q, --quiet Do not print any output
-v, --verbose... Use verbose output
--color <COLOR_CHOICE> Control colors in output [default: auto] [possible values: auto, always, never]
--native-tls Whether to load TLS certificates from the platform's native certificate store [env: UV_NATIVE_TLS=]
--offline Disable network access, relying only on locally cached data and locally available files
--no-progress Hides all progress outputs when set
--config-file <CONFIG_FILE> The path to a `uv.toml` file to use for configuration [env: UV_CONFIG_FILE=]
--no-config Avoid discovering configuration files (`pyproject.toml`, `uv.toml`) in the current directory, parent directories, or user configuration directories [env: UV_NO_CONFIG=]
-h, --help Print help
-V, --version Print version
Use `uv help add` for more details.
```
## Summary
`uv tree` will now filter to the current platform by default. You can
pass `--universal` to show the entire tree.
Closes https://github.com/astral-sh/uv/issues/5760.
## Summary
It's fine for this to be in the cache, I think, since we don't
necessarily need to colocate it with the Python directory.
Closes https://github.com/astral-sh/uv/issues/5747.
## Summary
After referring to https://github.com/astral-sh/uv/pull/5637 and doing
additional testing.
The default value in a stable state seems more reasonable to be
``only-system``. ``managed`` in preview.
```
cpython-3.11.9-windows-x86_64-none C:\Users\name\AppData\Local\Programs\Python\Python311\python.exe
cpython-3.10.14-windows-x86_64-none C:\Users\name\AppData\Roaming\uv\data\python\cpython-3.10.14-windows-x86_64-none\install\python.exe
cpython-3.10.11-windows-x86_64-none C:\Users\name\AppData\Local\Programs\Python\Python310\python.exe
cpython-3.9.19-windows-x86_64-none C:\Users\name\AppData\Roaming\uv\data\python\cpython-3.9.19-windows-x86_64-none\python.exe
```
test on uv 0.2.33 (build from
257007ccaf)
### Stable version
``uv venv -p 3.10`` is ``3.10.11`` (System Python)
``uv venv -p 3.9`` is ``No interpreter found``(3.9.19 for managed
Python)
``uv venv -p 3.9 --python-preference only-system`` is ``No interpreter
found``(fail)
``uv venv -p 3.9 --python-preference only-managed`` is
``3.9.19``(success)
Do not use managed Python, only use the system Python, so it can be
determined as ``only-system``.
### Preview mode
**Note:** ``3.10.14`` is managed python, ``3.10.11`` is system python.
``uv venv -p 3.11 --preview`` is ``3.11.9`` (System Python)
``uv venv -p 3.10 --preview`` is ``3.10.14``
``uv venv -p 3.10 --preview --python-preference only-managed`` is
``3.10.14``
``uv venv -p 3.10 --preview --python-preference managed`` is ``3.10.14``
``uv venv -p 3.10 --preview --python-preference system`` is ``3.10.11``
``venv -p 3.10 --preview --python-preference only-system`` is
``3.10.11``
Prioritize the managed Python and then select the system Python, so it
can be determined as ``managed``.
-----
fixed#5754
## Test Plan
Run website in local.
## Summary
Right now, if you have a `requirements.txt` with a pre-release, but the
`requirements.in` does not have a pre-release marker for that dependency
we drop the pre-release. (In the selector, we end up returning
`AllowPrerelease::IfNecessary`, the default.)
I played with a few ways of solving this... The first was to remove that
guard altogether. But if we do that,
`universal_transitive_disjoint_prerelease_requirement` fails (we use
`1.17.0rc1` in both forks, when it should only apply to one of the two).
The second was to do that, but also avoid pushing pre-releases as
preferences when we solve a fork. But then
`universal_disjoint_prereleases` fails, because we return a different
pre-release in each fork.
Finally, I settled on allowing existing pre-releases in forks if they
have no markers on them, i.e., they are "global" preferences. I believe
this is true IFF the preference came from an existing lockfile.
Closes https://github.com/astral-sh/uv/issues/5729.
## Summary
If _both_ nodes in the derivation tree are proxies, we need to remove
the _entire_ node. So, the function now returns an `Option<Tree>`
instead of taking `&mut Tree`.
Closes https://github.com/astral-sh/uv/issues/5618.
To enforce the 100 character line limit in markdown files introduced in
https://github.com/astral-sh/uv/pull/5635, and to automate the
formatting of markdown files, i've added prettier and formatted our
markdown files with it.
I've excluded the changelog and the generated references documentation
from this for having too many changes, but we can also include them.
I'm not particular on which style we use. My main motivations are
(major) not having to reflow markdown files myself anymore and (minor)
consistence between all markdown files. I've chosen prettier for similar
reason as we chose black, it's a single good style that's automated and
shared in the community. I do prefer prettier's style of not breaking
inside of a link name though.
This PR is in two parts, the first adds prettier to CI and documents
using it, while the second actually formats the docs. When merge
conflicts arise, we can drop the last commit and regenerate it with `npx
prettier --prose-wrap always --write BENCHMARKS.md CONTRIBUTING.md
README.md STYLE.md docs/*.md docs/concepts/**/*.md docs/guides/**/*.md
docs/pip/**/*.md`.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Partially resolves#5561. Haven't added overrides support yet but I can
add it tomorrow if the current approach for constraints is ok.
## Test Plan
`cargo test`
Manually checked trace logs after changing the constraints.
## Summary
Gives you a nice error message if you attempt to sync with, e.g., `-p
3.8` when that version is supported by at least one workspace member,
but your project's minimum requirement is `>=3.12`
Closes https://github.com/astral-sh/uv/issues/5662.
## Summary
I think it's reasonable to only sync the affected group, e.g., `uv add`
on its own should not require syncing all extras.
Closes https://github.com/astral-sh/uv/issues/4418.
Part of #4454
e.g. for `uv help pip compile`
```
Python options:
--python <PYTHON>
The Python interpreter against which to compile the requirements.
By default, uv uses the virtual environment in the current working directory or any parent
directory, falling back to searching for a Python executable in `PATH`. The `--python`
option allows you to specify a different interpreter.
Supported formats:
- `3.10` looks for an installed Python 3.10 using `py --list-paths` on Windows, or
`python3.10` on Linux and macOS.
- `python3.10` or `python.exe` looks for a binary with the given name in `PATH`.
- `/home/ferris/.local/bin/python3.10` uses the exact Python at the given path.
-p, --python-version <PYTHON_VERSION>
The minimum Python version that should be supported by the resolved requirements (e.g., `3.8` or `3.8.17`).
If a patch version is omitted, the minimum patch version is assumed. For example, `3.8` is mapped to `3.8.0`.
--python-preference <PYTHON_PREFERENCE>
Whether to prefer using Python installations that are already present on the system, or those that are downloaded and installed by uv
Possible values:
- only-managed: Only use managed Python installations; never use system Python installations
- managed: Prefer managed Python installations over system Python installations
- system: Prefer system Python installations over managed Python installations
- only-system: Only use system Python installations; never use managed Python installations
--python-fetch <PYTHON_FETCH>
Whether to automatically download Python when required
Possible values:
- automatic: Automatically fetch managed Python installations when needed
- manual: Do not automatically fetch managed Python installations; require explicit installation
```
## Summary
In #5494, I made breaking changes to the tool receipt format. This would
break existing tools for all users. This PR makes the change
backwards-compatible by supporting deserialization for the deprecated
format.
Closes https://github.com/astral-sh/uv/issues/5680.
## Test Plan
Beyond the automated tests, you can run `cargo run tool list` on your
existing machine.
Before:
```
warning: `uv tool list` is experimental and may change without warning
warning: Ignoring malformed tool `black` (run `uv tool uninstall black` to remove)
warning: Ignoring malformed tool `poetry` (run `uv tool uninstall poetry` to remove)
warning: Ignoring malformed tool `ruff` (run `uv tool uninstall ruff` to remove)
```
After:
```
warning: `uv tool list` is experimental and may change without warning
black v0.1.0
- black
poetry v1.8.3
- poetry
ruff v0.0.60
- ruff
```
## Summary
When we add a new optional group in `uv add`, we never to update the
`pyproject.toml` before locking. Otherwise, we use the stale
`pyproject.toml` and omit the optional group.
Closes https://github.com/astral-sh/uv/issues/5687.
## Summary
Fixes a bug in #5494. The `RequirementSourceWire` representation was
ambiguous, and so the order of the fields meant that all variants were
mapped to `Registry` when deserializing. (So the snapshots were right,
but behaviors were wrong.)
## Summary
This could still be made more robust, but it's not critical, since you
can always `--force`. It's good to handle this case, though, since we
have an explicit error for it.
Closes https://github.com/astral-sh/uv/issues/5490.
It transpires that detecting the directory a script was sourced from is
non-trivial across `bash`, `ksh` and `zsh`.
The previous version was a one-liner and supported `bash` and `zsh` but
not `ksh`.
It is possible to keep the one-liner and add `ksh` support, but that is
mutually-exclusive with `zsh`.
Therefore, the only way to square this circle is to add an `if` block. A
silver lining here is that although longer, the script is probably
easier to follow as there is less code-golfing going on.
## Summary
The current receipt doesn't capture quite enough information. For
example, it doesn't differentiate between editable and non-editable
requirements. This PR instead uses the full `Requirement` type. I think
we should use a custom representation like we do in the lockfile, but
I'm just using the default representation to demonstrate the idea.
## Summary
As-is, if you have a workspace with mixed `requires-python`
requirements, resolution will _never_ succeed, since we'll use the union
as the `requires-python` bound (i.e., take the lowest value), and fail
when we see the package that only supports some more narrow range.
This PR modifies the behavior to take the intersection (i.e., the
highest value), so if you have one package that supports Python 3.12 and
later, and another that supports Python 3.8 and later, we lock for
Python 3.12. If you try to sync or run with Python 3.8, we raise an
error, since the lockfile will be incompatible with that request.
Konsti has a write-up in https://github.com/astral-sh/uv/issues/5594
that outlines what could be a longer-term strategy.
Closes https://github.com/astral-sh/uv/issues/5578.
By resolving for each fork from the lockfile individually and by adding
using preferences for the current fork, we solve the instability #5180.
I've tested the locally and will add the packse test scenarios upstack.
Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198
The comment in the code explains the bulk of this:
```rust
// We previously computed this heuristic freshness lifetime by
// looking at the difference between the last modified header and
// the response's date header. We then asserted that the cached
// response ought to be "fresh" for 10% of that interval.
//
// It turns out that this can result in very long freshness
// lifetimes[1] that lead to uv caching too aggressively.
//
// Since PyPI sets a max-age of 600 seconds and since we're
// principally just interacting with Python package indices here,
// we just assume a freshness lifetime equal to what PyPI has.
//
// Note though that a better solution here is for the index to
// support proper HTTP caching headers (ideally Cache-Control, but
// Expires also works too, as above).
```
We also remove the `heuristic_percent` field on `CacheConfig`. Since
that's actually part of the cache itself, we bump the simple cache
version.
Finally, we add some more `trace!` calls that should hopefully make
diagnosing issues related to the freshness lifetime a bit easier in the
future.
Fixes#5351
## Summary
Given a fork like:
```
pylint < 3 ; sys_platform == 'darwin'
pylint > 2 ; sys_platform != 'darwin'
```
Solving the top branch will typically yield a solution that also
satisfies the bottom branch, due to maximum version selection (while the
inverse isn't true).
To quote an example from the docs:
```rust
// If there's no difference, prioritize forks with upper bounds. We'd prefer to solve
// `numpy <= 2` before solving `numpy >= 1`, since the resolution produced by the former
// might work for the latter, but the inverse is unlikely to be true due to maximum
// version selection. (Selecting `numpy==2.0.0` would satisfy both forks, but selecting
// the latest `numpy` would not.)
```
Closes https://github.com/astral-sh/uv/issues/4926 for now.
## Summary
First part of: https://github.com/astral-sh/uv/issues/4926. We should
solve forks that _don't_ expand the world of supported versions (e.g.,
`python_version >= '3.11'` enables us to select new packages, since we
narrow the supported version range).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
`uv venv` should support adopting python version specified in
`requires-python` from `pyproject.toml`. This allows customization on
the venv setup when syncing from python project.
Closes https://github.com/astral-sh/uv/issues/5552.
It also serves as a workaround to close
https://github.com/astral-sh/uv/issues/5258.
## Test Plan
<!-- How was it tested? -->
1. Run `uv venv` in folder with `pyroject.toml` specifying
`requries-python = "<3.10"`. Python 3.9 is selected for venv.
2. Change to `requries-python = "<3.11"` and run `uv venv` again. Python
3.10 is selected now.
3. Switch to a folder without `pyproject.toml` then run `uv venv`.
Python 3.12 is selected now.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Collapses the previous default into "managed" and makes the "managed"
behavior match "installed". People should use "only-managed" if they
want that behavior, it seems overly complicated otherwise.
## Summary
This PR deprecates the `--isolated` flag. The treatment varies across
the APIs:
- For non-preview APIs, we warn but treat it as equivalent to
`--no-config`.
- For preview APIs, we warn and ignore it, with two exceptions...
- For `tool run` and `run` specifically, we don't even warn, because we
can't differentiate the command-specific `--isolated` from the global
`--isolated`.
## Summary
The culmination of #4730. We now have `uv run --isolated` which always
uses a fresh environment (but includes the workspace dependencies as
needed). This enables you to test with strict isolation (e.g., `uv run
--isolated -p foo` will ensure that `foo` is unable to import anything
that isn't an actual dependency).
Closes#5430.
## Summary
This PR gets rid of the global `--isolated` flag (which serves a bunch
of independent responsibilities right now) on `uv tool run` in favor of
a dedicated `--isolated` flag, which tells uv to avoid re-using an
existing tool environment for this invocation. We'll add the same thing
to `uv run`, to avoid using the base project environment.
This will become a bit clearer in #5466, when we deprecate the
`--isolated` flag on the preview APIs.
## Summary
The idea here is that we hide all resolver output (the grayed out
resolver messages, plus the list of environment modifications) by
default in `uv run` and `uv tool run`. You can pass `--show-resolution`
to re-enable them.
Closes https://github.com/astral-sh/uv/issues/5458.
## Summary
Right now, `--isolated` is read from `uv run` and `uv init` to avoid
discovering the current workspace (or project). This PR moves that
behavior to a dedicated `--no-workspace` flag for `uv init`, and
`--no-project` for `uv run`. They could use the same flag, but
`--no-project` feels confusing for `uv init`, and `--no-workspace` seems
confusing for `uv run` (especially so once you read the documentation,
where we refer to the thing you're omitting as the project).
Closes https://github.com/astral-sh/uv/issues/5429.
This PR represents a different approach to marker propagation in an
attempt to unblock #4640. In particular, instead of propagating markers
when forks are created, we wait until resolution is complete to
propagate all markers to all dependencies in each fork. This ends up
being both more robust (we should never miss anything) and simpler to
implement because it doesn't require mutating a `PubGrubPackage` (which
was pretty annoying). I think the main downside here is that this can
sometimes add markers where they aren't needed.
This actually winds up making quite a few snapshot changes. I went
through each of them. Some of them look like legitimate bug fixes. Some
of them look like superfluous additions. And some of them look like they
would be removed if we had perfect marker normalization. But I don't
think any of the changes are _wrong_.
## Summary
If we just created an entrypoint script, we can of course set the
permissions (we just created it). However, if we're copying from the
cache, we might _not_ own the file. In that case, if we need to change
the permissions (we shouldn't, since the script is likely already
executable -- we set the permissions when we unzip, but I guess they
could _not_ be properly set in the zip itself), we have to copy it.
Closes https://github.com/astral-sh/uv/issues/5581.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
use windows-sys bindings maintained by microsoft devs. winapi didn't has
any updates for more than 3 years
## Test Plan
cargo test. it failed locally because I don't have Python 3.12 installed
## Summary
uv run --directory <path> means that one doesn't have to change to a
project's directory to run programs from it. It makes it possible to use
projects as if they are tool installations.
To support this, first the code reading .python-version was updated so
that
it can read such markers outside the current directory. Note the minor
change this causes (if I'm right), described in the commit.
## Test Plan
One test has been added.
## --directory
Not sure what the name of the argument should be, but it's following uv
sync's directory for now.
Other alternatives could be "--project". Uv run and uv tool run should
probably find common agreement on this (relevant for project-locked
tools).
I've implemented this same change in Rye, some time ago, and then we
went
with --pyproject `<`path to pyproject.toml file`>`. I think using
pyproject.toml file path and not directory was probably a mistake, an
overgeneralization one doesn't need.
Every packse version update is currently causing a huge diff (the size
of the `lock_scenarios.rs` diff in this PR). By redacting the version
from the snapshots, we will only have the actual change in the diff and
not the redundant version change noise.
The second commit moves all remaining packse url arg values to
`common/mod.rs`, which acts as a single source of truth for the packse
version.
## Summary
The package was being installed as editable, but it wasn't marked as
such in `uv pip list`, as the `direct-url.json` was wrong.
Closes https://github.com/astral-sh/uv/issues/5543.
## Summary
The idea here is similar to what we do for wheels: we create the
`CachedEnvironment` in the `archive-v0` bucket, then symlink it to its
content-addressed location. This ensures that we can always recreate
these environments without concern for whether anyone else is accessing
them.
Part of the challenge here is that we want the virtual environments to
be relocatable, because we're now building them in one location but
persisting them in another. This requires that we write relative (rather
than absolute) paths to scripts and entrypoints. The main risk with
relocatable virtual environments is that the scripts and entrypoints
_themselves_ are not relocatable, because they use a relative shebang.
But that's fine for cached environments, which are never intended to
leave the cache.
Closes https://github.com/astral-sh/uv/issues/5503.
## Summary
Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:
* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
`.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.
The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).
Fixes: #3863
## Test Plan
* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
## Summary
This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.
Part of: https://github.com/astral-sh/uv/issues/5263.
## Test Plan
For example:
```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
## Summary
Closes#2187.
The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.
It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.
## Test Plan
Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`
(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.
## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'
Usage: uv.exe pip list --editable
For more information, try '--help'.
```
Consider the following packse scenario:
```toml
[root]
requires = [
"a>=1.0.0 ; python_version < '3.10'",
"a>=1.1.0 ; python_version >= '3.10'",
"a>=1.2.0 ; python_version >= '3.11'",
]
[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```
On current `main`, this produces a dependency on `a` that looks like
this:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```
But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a" },
]
```
That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.
The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.
While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.
For the packse scenario test: https://github.com/astral-sh/packse/pull/206
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.
This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.
In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.
So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.
This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.
This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.
Fixes#5086
Partially addresses #4732
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
## Summary
After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.
This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.
Closes https://github.com/astral-sh/uv/issues/5459.
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
## Summary
It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.
This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.
Closes https://github.com/astral-sh/uv/issues/5424.
## Summary
This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).
Now that we support persistent configuration, users should add:
```toml
[tool.uv]
reinstall = ["foo"]
```
If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).
Closes https://github.com/astral-sh/uv/issues/5420.
Two changes split out from the instability work:
* Break `ResolutionGraph::from_state` into methods before adding new
logic to it.
* `ResolutionGraph`: Convert `NodeKey` type to `PackageRef` struct:
Another small refactoring to make subsequent changes easier.
No functional changes.
@BurntSushi I hope this doesn't interfere with your work too much, the
`PackageRef` should at least make debugging panics here easier.
In preparation for the preferences changes with forking, change the
method structure in `CandidateSelector`. Split out into its own PR to
avoid merge conflicts with main. No functional changes.
Consider these requirements from pylint 3.2.5:
```
Requires-Dist: dill >=0.3.6 ; python_version >= "3.11"
Requires-Dist: dill >=0.3.7 ; python_version >= "3.12"
```
We will split on the python version, but then we may pick a version of
`dill` that's `>=0.3.7` in both branches and also have an otherwise
identical resolution in both forks. In this case, we merge both forks
and store only their conjoined markers.
## Summary
Normalize the order of marker expressions on construction. This removes
the distinction between expressions like `os_name == 'Linux'` vs.
`'Linux' == os_name` throughout the codebase. One caveat here is that
the `in` operator does not have a direct inverse, so we introduce
`MarkerOperator::Contains` to handle that case.
I wanted to land this smaller change before some more intrusive changes
as it simplifies the existing code quite a bit.
## Summary
Prior to this change, the resolver would panic if we ran with
`--offline` and `--no-deps` and we had cached metadata for a _package_
(i.e., the versions) but no cached metadata for the _distribution_
(i.e., the specific wheel), since we weren't validating that the
returned metadata in the `--no-deps` case was actually successful. (We
need metadata, even for `--no-deps`, so that we can validate extras.)
## Test Plan
The added test panics on the previous branch.
## Summary
Users can now run `uv cache prune --ci` (open to feedback on the name of
that flag) to remove all pre-built wheels from the cache, leaving behind
zipped, built wheels (which tend to be the most expensive assets to
re-create). This should greatly increase cache performance in CI
environments, since uploading unzipped wheels can actually hurt
performance if you're persisting the uv cache.
Closes https://github.com/astral-sh/uv/issues/5282.
The test asserts that 28 files were removed. But on my system, 27 files
are removed.
This PR is first about debugging what the difference is (since CI
presumably passes with the status quo snapshot). And then I'm thinking
the right way to fix the test failure is with a filter that replaces the
specific number of files removed (limited to what we know to be
correct) with a placeholder.
## Summary
This is surprisingly complex because we need to decide what happens if
you run `uv run` from within a hidden folder, etc. For now, I did the
simplest thing: we just ignore workspace members that are hidden
directories if they lack a `pyproject.toml`, so you can still include
hidden members, they're just ignored if they don't seem to be projects.
Closes https://github.com/astral-sh/uv/issues/5403.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Implements #5340
## Test Plan
<!-- How was it tested? -->
## Summary
This is a bit simpler than #5333, but seems to work in my testing on
macOS and Windows. It's based on implementations that I found in
[Pixi](36f1bb297d/src/cli/exec.rs (L99))
and
[Wasmer](49e60af8df/lib/wasix/src/state/builder.rs (L1058)).
Closes https://github.com/astral-sh/uv/issues/5257.
## Test Plan
On both macOS and Windows:
- `cargo run -- tool run --from jupyterlab jupyter-lab` -- hit Ctrl-C;
verify that the process exits and the terminal is left in a good state.
- `cargo run -- run python` -- hit Ctrl-C; verify that the process does
_not_ exit, but does on Ctrl-D.
## Summary
Closes https://github.com/astral-sh/uv/issues/5359.
## Test Plan
Unfortunately, the only packages I know of that use this are Ruff and
uv, and both are too heavy to install in a recurring test, so:
`uv tool install hatch==1.12.0 --with uv==0.2.27 --force
--link-mode=symlink`
> DEBUG Found `cpython-3.12.1-macos-aarch64-none` at
`/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
Instead of `<implementation> <version>`
> DEBUG Found cpython 3.12.1 at `/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
konsti
Ahmed Ilyas
Udi Oron
Zanie Blue
adisbladis
Charlie Marsh
Jo
Eric Mark Martin
Daniel Bruckner
Andrew Gallant
Alex Willmer
Jp
Jp
Li-Lun Lin
Skyler Hawthorne
Matthijs Kok
Jon Åslund
Ben Beasley
Ed Morley
Hristo Filaretov
renovate[bot]
Shantanu
Micha Reiser
Owen Brooks
Shane Kennedy
Mark Ryan
Aditya Pratap Singh
Adam Björnberg
Noé Rubinstein
Robert Deaton
Tim de Jager
samypr100
Pietro Zambelli
Simon Willison
Tim Chan
Vivien Maisonneuve
Magnus Markling
Nathan McDougall
Andrey Bozhko
tfsingh
Amjith Ramanujam
kkdamowang
Alexander Gherm
InSync
Vini Brasil
David Bern
Tim Hatch
github-actions[bot]
Ian Paul
bluss
Amos Wenger
Max Friedrich
Jiahao Yuan
Noam Teyssier
Trevor Manz
Pavel Dikov
Brandon W Maister
Jacob Coffee
trag1c
Kemal Akkoyun
sobolevn
Krishnan Chandra
Seth Morton
Luca Bruno
Aarni Koskela
Liam
Topher Anderson
Bas Schoenmaeckers
Huang, Hong-Chang
Abdó Roig-Maranges
You Jiacheng
kyoto7250
Mikko Leppänen
Mathieu Kniewallner
Dan Watson
Hanlu
Frost Ming
Michał Górny
Ibraheem Ahmed
Bartosz Sławecki
Soof Golan
Janosh Riebesell
Alex Gaynor
eth3lbert
Garth Kidd
Leiser Fernández Gallo
Michal Čihař
Chao Ning
Vikas
Chan Kang
my1e5
leaf-soba
Di-Is
T-256
Thomas Quillan
Sofie Van Landeghem
Severen Redwood
FishAlchemist
Branch Vincent
Delgan
Vigilans
Maksim Bondarenkov