## Summary
Historically, we've allowed the use of wheels that were downloaded from
PyPI even when the user passes `--no-binary`, if the wheel exists in the
cache. This PR modifies the cache lookup code such that we respect
`--no-build` and `--no-binary` in those paths.
Closes https://github.com/astral-sh/uv/issues/2154.
## Summary
My last changes (#6616) used by mistake == instead of !=.
😥 Making values currently never trimmed despite
what we wanted.
Values should now be trimmed if needed.
Also removes the trim of the header name, because if a header contains
spaces, the header will be skipped by the mailparse crate in the first
place.
## Test Plan
- A unit test has been added to validate that we correctly trim values.
- A unit test has been added to validate the header names containing
spaces are skipped.
## Summary
I have a workflow where I want use `uv` as a dependency solver only, and
manage my environments with external tooling (Nix).
## Test Plan
Manually tested. Automated testing seems excessive for such a trivial
change.
## Problems
It's still not as useful as I'd like it to be.
`uv` uncondtionally creates a virtual environment, something I would
expect that `--no-sync` should disable.
This looks a bit more tricky to achieve and I'm not sure about how to
best structure it.
## Summary
This is another attempt using `module: bool` instead of `module:
Option<String>` following #7322.
The original PR can't be reopened after a force-push to the branch, I've
created this new PR.
Resolves#6638
## Summary
This PR fixes#7733. According to [CPython documentation on
`sys.stdout`](https://docs.python.org/3.12/library/sys.html#sys.stdout),
when `stdout`/`stderr` is non-character device like pipe, the encoding
will be set to system locale on windows. However, on the Rust side
`stdout_reader` and `stderr_reader` expect them to be encoded in UTF-8
and will fail when child process write non-ASCII character to
stdout/stderr, e.g., build directory name containing non-ASCII
character.
Both
[CPython3](https://docs.python.org/3.12/using/cmdline.html#envvar-PYTHONIOENCODING)
and [PyPy](https://doc.pypy.org/en/default/man/pypy3.1.html#environment)
support environment variable `PYTHONIOENCODING`. When it is set to
`utf-8`, python will use UTF-8 encoding for `stdin`/`stdout`/`stderr`.
Since `stdin` is not used by the spawned python process and we expect
`stdout`/`stderr` to use UTF-8, this fix should work as expected.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
I only tested it on my computer with CPython 3.12 and 3.7. With the fix
applied I confirmed that [the case I
described](https://github.com/astral-sh/uv/issues/7733#issuecomment-2380416093)
is fixed.
I'm using Windows 11 with system locale set to code page 936.
## Summary
Small follow up to https://github.com/astral-sh/uv/pull/7724
## Test Plan
`cargo test`
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Resolves#7705
## Test Plan
`cargo test` and tested locally.
The snapshots were unstable due to the packages being built in a
non-deterministic order, so I used the quiet flag to suppress the
output.
Another question is whether we should label the build output to indicate
which package it belongs to?
## Summary
Adds a helpful context message when `uvx` is run without arguments
To clarify, it is displaying the installed tools.
This addresses confusion, such as the one highlighted in issue #7348,
by making the output more user-friendly and informative.
Related #4024
## Test Plan
Updated the test snapshots to include the new output.
Running the tests locally with `cargo nextest run` confirms that the
tests pass.
The CI pipeline should also pass.
### Manuel Testing
**uvx**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uvx
Provide a command to invoke with `uvx <command>` or `uvx --from <package> <command>`.
The following tools are already installed:
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
See `uvx --help` for more information.
```
**uv tool list**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uv tool list
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
```
**uv tool run**
```shell
# Make sure you have the updated version of uv installed on your path.
# cargo install --path ./crates/uv --force
❯ uv tool run
Provide a command to invoke with `uv tool run <command>` or `uv tool run --from <package> <command>`.
The following tools are already installed:
black v24.8.0
- black
- blackd
ruff v0.6.7
- ruff
See `uv tool run --help` for more information.
```
---
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
---------
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
## Summary
Similiar to `cargo init --vcs <VCS>`, this PR adds the `--vcs <VCS>`
flag for `uv init`, allowing to create a version control system during
initialization. By default, `uv init` will create a Git repository if
the `--vcs` flag is not provided. Use `--vcs none` to disable this
feature.
Currently, only Git is supported. While Cargo also supports hg, pijul,
and fossil, this initial PR only includes Git. We can add more later if
there are any user requests.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This PR adds support for ```uv init --script```, as defined in issue
#7402 (started working on this before I saw jbvsmo's PR). Wanted to
highlight a few decisions I made that differ from the existing PR:
1. ```--script``` takes a path, instead of a path/name. This potentially
leads to a little ambiguity (I can certainly elaborate in the docs,
lmk!), but strictly allowing ```uv init --script path/to/script.py```
felt a little more natural than allowing for ```uv init --script path/to
--name script.py``` (which I also thought would prompt more questions
for users, such as should the name include the .py extension?)
2. The request is processed immediately in the ```init``` method,
sharing logic in resolving which python version to use with ```uv add
--script```. This made more sense to me — since scripts are meant to
operate in isolation, they shouldn't consider the context of an
encompassing package should one exist (I also think this decision makes
the relative codepaths for scripts/packages easier to follow).
3. No readme — readme felt a little excessive for a script, but I can of
course add it in!
---------
Co-authored-by: João Bernardo Oliveira <jbvsmo@gmail.com>
uv will soon support both a build frontend (`uv build`) and a build
backend (`build-system = "uv"`). To avoid the name clash, I'm renaming
the `uv-build` crate to `uv-build-frontend`. In a follow-up PR, I will
add a `uv-build-backend` crate with the build backend implementation.
This PR adds support for upgrading the build environment of tools with
the addition of a ```--python``` argument to ```uv upgrade```, as
specified in #7471.
Some things to note:
- I added support for individual packages — I didn't think there was a
good reason for ```--python``` to only apply to all packages
- Upgrading with ```--python``` also upgrades the package itself — I
think this is fair as if a user wants to _strictly_ switch the version
of Python being used to build a tool's environment they can use ```uv
install```. This behavior can of course be modified if others don't
agree!
Closes https://github.com/astral-sh/uv/issues/6297.
Closes https://github.com/astral-sh/uv/issues/7471.
#7226 modified the check to skip prefetching of source dists without
proper minimum-version bounds, and wound up flipping the boolean
expression. This change flips the some/none expression so that the
intended skip happens as expected.
Fixes#7680.
Closes#7118
This only really affects managed interpreters, as we exclude alternative
Python implementations from the search path during the
`VersionRequest::executable_names` part of discovery.
## Summary
Random, but I noticed that we can remove a ton of serialize and
deserialize derives by using `rkyv` for the flat-index caches. (We
already use `rkyv` for these same structs in the registry cache.)
This PR adds some additional sanity checking on resolution graphs to
ensure we can never install different versions of the same package into
the same environment.
I used code similar to this to provoke bugs in the resolver before the
release, but it never made it into `main`. Here, we add the error
checking to the creation of `ResolutionGraph`, since this is where it's
most convenient to access the "full" markers of each distribution.
We only report an error when `debug_assertions` are enabled to avoid
rendering `uv` *completely* unusuable if a bug were to occur in a
production binary. For example, maybe a conflict is detected in a marker
environment that isn't actually used. While not ideal, `uv` is still
usable for any other marker environment.
Closes#5598
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
closes#4828
First iteration for an implementation. I need to add more tests but
wanted your opinion on the implementation first.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Currently tested using the following command but will add tests shortly:
```console
D:\repo\uv> cargo run venv -p 3.13t && .venv\Scripts\python.exe
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
Running `target\debug\uv.exe venv -p 3.13t`
Using Python 3.13.0rc1 interpreter at: C:\Users\bschoen\AppData\Local\Programs\Python\Python313\python3.13t.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e78, Jul 31 2024, 21:06:58) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
There are two parts to this.
The first is a restructuring and refactoring. We had some debt around
expected executable name generation, which we address here by
consolidating into a single function that generates a combination of
names. This includes a bit of extra code around free-threaded variants
because this was written on top of #7431 — I'll rebase that on top of
this.
The second addresses some bugs around alternative implementations.
Notably, `uv python list` does not discovery executables with
alternative implementation names. Now, we properly generate all of the
executable names for `VersionRequest::Any` (originally implemented in
https://github.com/astral-sh/uv/pull/7508) to properly show all the
implementations we can find:
```
❯ cargo run -q -- python list --no-python-downloads
cpython-3.12.6-macos-aarch64-none /opt/homebrew/opt/python@3.12/bin/python3.12 -> ../Frameworks/Python.framework/Versions/3.12/bin/python3.12
cpython-3.11.10-macos-aarch64-none /opt/homebrew/opt/python@3.11/bin/python3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/python3.11
cpython-3.9.6-macos-aarch64-none /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.9/bin/python3
pypy-3.10.14-macos-aarch64-none /opt/homebrew/bin/pypy3 -> ../Cellar/pypy3.10/7.3.17/bin/pypy3
```
While doing both of these changes, I ended up changing the priority of
interpreter discovery slightly. For example, given that the executables
are in the same directory, do we query `python` or `python3.10` first
when you request `--python 3.10`? Previously, we'd check `python3.10`
but I think that was an incorrect optimization. I think we should always
prefer the bare name (i.e. `python`) first. Similarly, this applies to
`python` and an executable for an alternative implementation like
`pypy`. If it's not compatible with the request, we'll skip it anyway.
We might have to query more interpreters with this approach but it seems
rare.
Closes https://github.com/astral-sh/uv/issues/7286 superseding
https://github.com/astral-sh/uv/pull/7508
- **Do not attempt to reflink directories on linux**
- **Refactor clone_recursive**
## Summary
On linux, reflink does not work on a directory. Currently, we first
attempt to reflink directory, and only if it fails with `AlreadyExists`
we attempt to reflink recursively.
This has the effect that, on linux, `uv pip install --link-mode=clone`
would always fall back to `copy`.
We resolve this by only attempting to reflink directories on macos. In
the process, we refactored `clone_recursive` in an attempt to make it
easier to reason about its logic.
## Test Plan
I tested that after this change, `uv pip install --link-mode=clone
numpy` would behave as expected in the following cases:
* linux, btrfs filesystem, venv on the same filesystem as cache
(correctly reflinked)
* linux, btrfs filesystem, venv on a different filesystem than cache
(fallback to copy)
I have not tested it on macos or windows, as I currently don't have
access to any macos or windows machines, unfortunately.
## Summary
If the `requires-python` bound expands, the space covered by
`resolution-markers` may no longer include all supported Python
versions. In such cases, we need to avoid reusing the forks (but we
_can_ reuse the preferred versions).
Closes https://github.com/astral-sh/uv/issues/7618.
## Summary
`uv run --project ./path/to/project` now uses the provided directory as
the starting point for any file discovery. However, relative paths are
still resolved relative to the current working directory.
Closes https://github.com/astral-sh/uv/issues/5613.
## Summary
`#[serde(flatten)]` has a disastrous effect on error messages: serde no
longer tells you which field errored, nor does it show it to you in the
diagnostic output.
Before:
```
warning: Failed to parse `pyproject.toml` during settings discovery:
TOML parse error at line 9, column 1
|
9 | [tool.uv]
| ^^^^^^^^^
invalid type: string "foo", expected a sequence
```
After:
```
warning: Failed to parse `pyproject.toml` during settings discovery:
TOML parse error at line 10, column 19
|
10 | extra-index-url = "foo"
| ^^^^^
invalid type: string "foo", expected a sequence
```
Closes https://github.com/astral-sh/uv/issues/7113.
## Summary
This reverts commit 3060fd22c0.
These are now _never_ shown to users, because `tracing` isn't set up at
that point. I'm going to try and improve the solution more holistically,
but this is better than the status quo.
Closes https://github.com/astral-sh/uv/issues/7573.
This enhances the hints generator in the resolver with some heuristic to
detect and warn in case of failures due to version mismatches on a local
package. Those may be the symptom of name conflict/shadowing with a
transitive dependency.
Closes: https://github.com/astral-sh/uv/issues/7329
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Because a problem was found with Powershell and combining the generated
completion scripts for uv and uvx, let's try separating uv and uvx
command completion scripts.
The generated powershell script template can be seen in clap_complete
source, and it starts with `using` directives, which makes it impossible
(apparently) to concatenate two of those script outputs.
As a side effect, this is available under `uv tool run
--generate-shell-completion` too.
Fixes#7482
## Test Plan
- `eval "$(cargo run --bin uvx -- --generate-shell-completion bash)"`
- Test Powershell
## Summary
The AGPL license is confusing some analyzers. This replaces pretix with
saleor which (similarly) is a web application.
Closes https://github.com/astral-sh/uv/issues/7566.
## Summary
`uv cache prune --ci` will remove the source distribution directory. If
we then need to build a _different_ wheel (e.g., you're building a
package that has Python minor version-specific wheels), we fail, because
we expect the source to be there.
Now, if the source is missing, we re-download it. It would be slightly
easier to just _ignore_ that revision, but that would mean we'd also
lose the already-built wheels -- so if you ran against many Python
versions, we'd continuously lose the cached data.
Closes https://github.com/astral-sh/uv/issues/7543.
## Test Plan
We can add tests, but they _need_ to build non-pure Python wheels, which
tends to be expensive...
For reference:
```console
$ cargo run venv --python 3.12
$ cargo run pip install mercurial==6.8.1 --verbose
$ cargo run cache prune --ci
$ cargo run venv --python 3.11
$ cargo run pip install mercurial==6.8.1 --verbose
```
I also did this with a local `.tar.gz` that I downloaded from PyPI.
## Summary
Both of these can contain rkyv data in their HTTP cache envelopes. As
such, the entries aren't readable by earlier versions of uv, and `uv
cache prune` can break. I should make `uv cache prune` robust to this,
but this feels safest.
## Summary
I think this is just inverted. It means that when we fail in
https://github.com/astral-sh/uv/issues/7553, we show a message for
"invalid Python implementation" (since there are some wheels that don't
match), but we should be showing "invalid platform", matching the order
of operations in our compatibility check.
Closes https://github.com/astral-sh/uv/issues/7553.
Adds display of the target path of the link (since the link filename
itself is basically static) and distinguishes between broken links and
missing files.
## Summary
Improve the description of override-dependencies based on the statement
in `concepts/resolution.md`: "As with constraints, overrides do not add
a dependency on the package and only take effect if the package is
requested in a direct or transitive dependency."
I tested it locally, `concepts/resolution.md` is correct. It would be
better to also include this in the Reference Chapter of the docs.
I think it's best practice to use a placeholder when we transform
something, and #7522 is having snapshot issues because this filter
conflicts with `with_filtered_virtualenv_bin`
As we support more complex Python discovery behaviors such as:
- #7431
- #7335
- #7300
`Any` is no longer accurate, we actually are looking for a reasonable
default Python version to use which may exclude the first one we find.
Separately, we need the idea of `Any` to improve behavior when listing
versions (e.g., #7286) where we do actually want to match _any_ Python
version. As a first step, we'll rename `Any` to `Default`. Then, we'll
introduce a new `Any` that actually behaves as we'd expect.
Recently, rkyv 0.8 was released. Its API is a fair bit simpler now for
higher level uses (like for us in `uv`) and results in us being able to
delete a fair bit of code. This also removes our last dependency on `syn
1.0`, and thus drops that dependency.
Performance (via testing on the `transformers` example) seems to remain
about the same, which is what was expected:
```
$ hyperfine -w5 -r100 'uv lock' 'uv-ag-rkyv-update lock'
Benchmark 1: uv lock
Time (mean ± σ): 55.6 ms ± 6.4 ms [User: 30.4 ms, System: 35.1 ms]
Range (min … max): 43.0 ms … 73.1 ms 100 runs
Benchmark 2: uv-ag-rkyv-update lock
Time (mean ± σ): 56.5 ms ± 7.2 ms [User: 30.5 ms, System: 36.3 ms]
Range (min … max): 39.1 ms … 71.5 ms 100 runs
Summary
uv lock ran
1.02 ± 0.18 times faster than uv-ag-rkyv-update lock
```
Closes#7415
This changes the structure of the hints generator in the resolver when
encountering solution errors, so that it re-uses a single output buffer
owned by the caller.
It avoids repeated allocations of a temporary buffer within each
recursive function call.
## Summary
See: https://github.com/astral-sh/uv/issues/7485. The test was using `uv
pip sync` which doesn't require fetching metadata, and the failure was
in fetching metadata.
## Summary
Closes https://github.com/astral-sh/uv/issues/7485.
## Test Plan
```
$ cargo run cache clean
$ cargo run venv
$ cargo run pip install django-allauth==0.51.0
$ cargo run venv
$ cargo run pip install django-allauth==0.51.0
```
This changes `uv tool install` behavior with regards to re-using
existing environments.
In particular, this replaces the existing version-matching logic with a
tighter one, enforcing
a same-interpreter match.
This allows to properly switch between system and managed interpreter,
at the cost of
more eagerly invalidating existing environments every time there is an
interpreter change.
Closes: https://github.com/astral-sh/uv/issues/7320
## Summary
This PR enables users to provide pre-defined static metadata for
dependencies. It's intended for situations in which the user depends on
a package that does _not_ declare static metadata (e.g., a
`setup.py`-only sdist), and that is expensive to build or even cannot be
built on some architectures. For example, you might have a Linux-only
dependency that can't be built on ARM -- but we need to build that
package in order to generate the lockfile. By providing static metadata,
the user can instruct uv to avoid building that package at all.
For example, to override all `anyio` versions:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]
[[tool.uv.dependency-metadata]]
name = "anyio"
requires-dist = ["iniconfig"]
```
Or, to override a specific version:
```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]
[[tool.uv.dependency-metadata]]
name = "anyio"
version = "3.7.0"
requires-dist = ["iniconfig"]
```
The current implementation uses `Metadata23` directly, so we adhere to
the exact schema expected internally and defined by the standards. Any
entries are treated similarly to overrides, in that we won't even look
for `anyio@3.7.0` metadata in the above example. (In a way, this also
enables #4422, since you could remove a dependency for a specific
package, though it's probably too unwieldy to use in practice, since
you'd need to redefine the _rest_ of the metadata, and do that for every
package that requires the package you want to omit.)
This is under-documented, since I want to get feedback on the core ideas
and names involved.
Closes https://github.com/astral-sh/uv/issues/7393.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
close#6272
## Test Plan
<!-- How was it tested? -->
As in https://github.com/astral-sh/uv/pull/6262
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
When syncing a lockfile, we need to respect credentials defined in the
`pyproject.toml`, even if they won't be used for resolution.
Unfortunately, this includes credentials in `tool.uv.sources`,
`tool.uv.dev-dependencies`, `project.dependencies`, and
`project.optional-dependencies`.
Closes https://github.com/astral-sh/uv/issues/7453.
## Summary
This PR adds support to include Python pre-releases when requesting
versions.
Check out the docs for commands that support the `Python` option:
```text
--python, -p python
The Python interpreter to use for the virtual environment.
```
At least the following scenarios are supported:
```bash
3.13.0a1
3.13b2
3.13rc4
313rc1
```
## Test Plan
I added a basic unit test to `uv/crates/uv-python/src/discovery.rs`. I
could have added more, but I have not discovered any relevant places.
CI passes
Note: I was unable to execute the entire test set locally. There were at
least some timeout issues (some tests took over 60 seconds).
========== output ===========
beta version
```bash
cargo run -- venv --python 3.13.0b3 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.20s
Running `target/debug/uv venv --python 3.13.0b3`
Using Python 3.13.0b3 interpreter at: /home/mikko/.pyenv/versions/3.13.0b3/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
````
release candidate
```bash
cargo run -- venv --python 3.13.0rc2 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.83s
Running `target/debug/uv venv --python 3.13.0rc2`
Using Python 3.13.0rc2 interpreter at: /home/mikko/.pyenv/versions/3.13.0rc2/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
```bash
cargo run -- venv --python 313rc2 ░▒▓ 94%
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.31s
Running `target/debug/uv venv --python 313rc2`
Using Python 3.13.0rc2 interpreter at: /home/mikko/.pyenv/versions/3.13.0rc2/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Generate shell completion for uvx.
Create a `uvx` toplevel command just for completion by combining `uv
tool uvx` (hidden alias for `uv tool run`) with global arguments. This
explicit combination is needed otherwise global arguments are missing
(if they are missing, clap debug assertions fail when `uv tool run`
arguments refer to global arguments in directives like conflicts with).
Fixes#7258
## Test Plan
- Tested using bash using `eval "$(cargo run --bin uv
generate-shell-completion bash)"`
## Summary
All the registry wheels were getting cached under
`index/b2a7eb67d4c26b82` rather than `pypi`, because we used
`IndexUrl::Url` rather than `IndexUrl::from`.
## Summary
It's very unlikely that retaining these is beneficial, since you tend to
partition the cache by platform anyway.
Closes https://github.com/astral-sh/uv/issues/7394.
## Summary
Since https://github.com/astral-sh/uv/pull/7208, this is now _always_
firing, for every directory, because the version gets normalized (e.g.,
`1.2.3` gets normalized to `1-2-3`, which never matches the parsed
version). pip doesn't warn here, I guess we won't either, because I
can't figure out a robust way to do this... We need to get the
non-normalized remainder after stripping the normalized package name,
but we strip the normalized package name from the normalized string, so
we only have a normalized remainder.
## Summary
Running `uv lock --no-sources` should still include dev dependencies,
since dev dependencies are defined separately from sources.
Closes https://github.com/astral-sh/uv/issues/7406.
We got user reports where users were confused about why they can't use
`[project.urls]` in `pyproject.toml` (i think that's from poetry?). This
PR adds a hint that (according to PEP 621), you need to set
`project.name` when using any `project` fields. (PEP 621 also requires
`project.version` xor `dynamic = ["version"]`, but we check that later.)
The intermediate parsing layer to tell apart syntax errors from schema
errors doesn't incur a performance penalty according to epage
(https://github.com/toml-rs/toml/issues/778#issuecomment-2310369253).
Closes#6419Closes#6760
closes#7365
Summary
This pull request adds support for additional file extension aliases in
the SourceDistExtension and ExtensionError enums. The newly supported
file extensions include .tbz, .tgz, .txz, .tar.lz, .tar.lzma. These
changes align the extensions supported by the SourceDistExtension with
those used in Python packaging tools, enhancing compatibility with a
broader range of source distribution formats.
Test Plan
should be added or updated to verify that the new extensions are
correctly recognized as valid source distributions and that errors are
correctly raised when unsupported extensions are provided.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Part of https://github.com/astral-sh/uv/issues/7007.
Settings documentation reference currently doesn't separate "project
metadata" and "configuration" options, implying that it's possible to
set things like `dev-dependencies` in `uv.toml` while it's not. This is
an attempt at better separating those options, by having 2 different
sections:
- `Project metadata`, that holds configuration that can only be set in
`pyproject.toml`
- `Configuration`, that holds configuration that can be set both in
`pyproject.toml` and `uv.toml`
Here are some screenshots to show what this looks like (note that I
don't have code highlighting in the right navigation, which makes them
clunky, as first item is always bigger because of the missing "span" --
I think that's because it's an `mkdocs-material` insider feature, since
I have the same thing on `main` branch):
- Right side navigation:
<img width="241" alt="Screenshot 2024-09-05 at 01 19 50"
src="https://github.com/user-attachments/assets/012f64a4-8d34-4e34-a506-8d02dc1fbf98">
<img width="223" alt="Screenshot 2024-09-05 at 01 20 01"
src="https://github.com/user-attachments/assets/0b0fb71d-c9c3-4ee3-8f6e-cf35180b1a99">
- An option from "Project metadata" section that only applies to
`pyproject.toml`:
<img width="788" alt="Screenshot 2024-09-05 at 01 20 11"
src="https://github.com/user-attachments/assets/64349fbb-8623-4b81-a475-d6ff38c658f1">
- An option from "Configuration" section that applies both to
`pyproject.toml` and `uv.toml`:
<img width="787" alt="Screenshot 2024-09-05 at 01 20 33"
src="https://github.com/user-attachments/assets/732e43d3-cc64-4f5a-8929-23a5555d4c53">
## Test Plan
Local run of the documentation.
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Updates the output of `uv export` to include the command that produced
it, similar to how `uv pip compile` does. This addresses #7159 - I had
this same itch today, figured it was a good time to dive in!
## Test Plan
All the export unit tests were updated to test the new output format.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
fix symbol error
## Test Plan
<!-- How was it tested? -->
Signed-off-by: liangmulu <liangmulu@outlook.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
It often reaches the GitHub API rate limit and shows error like `error:
HTTP status client error (403 Forbidden) for url
(https://api.github.com/repos/astral-sh/uv/releases)` when running `uv
self update`.
To bypass this rate limit issue, allow user to pass a GitHub token via
`--token` or `UV_GITHUB_TOKEN` env.
## Test Plan
<!-- How was it tested? -->
---------
Signed-off-by: Frost Ming <me@frostming.com>
Summary
This pull request fixes a typo in the --build-constraints flag, which
should be singular (--build-constraint). This update ensures consistency
across the documentation and prevents potential confusion for users.
Closes#7315
## Test Plan
The change was verified by reviewing the relevant documentation files
where the flag is referenced. No functional code changes were made, so
no additional testing is required beyond confirming the documentation
update.
## Tested
The change was tested by visually inspecting the updated documentation
to confirm that the typo has been corrected
## Summary
This is arguably breaking, arguably a bug... Today, if project A depends
on project B, and you install A with dev dependencies enabled, you also
get B's dev dependencies. I think this is incorrect. Just like you
shouldn't be importing B's dependencies from A, you shouldn't be using
B's dev dependencies when developing on A.
Closes#7310.
Similar to our semantics for packages with pre-release versions.
We will not use prerelease versions unless there are only prerelease
versions available, a specific version is requested,
or the prerelease version is found in a reasonable source (active
environment, explicit path, etc. but not `PATH`).
For example, `uv python install 3.13 && uv run python --version` will no
longer use `3.13.0rc2` unless that is the only Python version available,
`--python 3.13` is used, or that's the Python version that is present in
`.venv`.
## Summary
This has been asked for a few times. There are risks that these checks
could be slow, but they're buyer-beware.
Closes https://github.com/astral-sh/uv/issues/7246.
## Summary
We have to call `to_dist` to get metadata while validating the lockfile,
but some of the distributions won't match the current platform -- and
that's fine!
## Summary
We need to apply the `--no-install` filters earlier, such that we don't
error if we only have a source distribution for a given package when
`--no-build` is provided but that package is _omitted_.
Closes#7247.
Following #7263 the 3.13.0rc2 releases are at the top of the download
list but we should not select them unless 3.13 is actually requested.
Prior to this, `uv python install` would install `3.13.0rc2`.
```
❯ cargo run -- python install --no-config
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv python install --no-config`
Searching for Python installations
Installed Python 3.12.6 in 1.33s
+ cpython-3.12.6-macos-aarch64-none
```
```
❯ cargo run -- python install --no-config 3.13
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv python install --no-config 3.13`
Searching for Python versions matching: Python 3.13
Installed Python 3.13.0rc2 in 1.18s
+ cpython-3.13.0rc2-macos-aarch64-none
```
## Summary
Replace the unmaintained `tokio-tar` crate with the `krata-tokio-tar`
fork. The latter just merged a fix necessary for the crate to work on
PowerPC, and has better chances of future maintenance.
Fixes#3423
## Test Plan
`cargo test`
This is preparatory work for the upload functionality, which needs to
read the METADATA file and attach its parsed contents to the POST
request: We move finding the `.dist-info` from `install-wheel-rs` and
`uv-client` to a new `uv-metadata` crate, so it can be shared with the
publish crate.
I don't properly know if its the right place since the upload code isn't
ready, but i'm PR-ing it now because it already had merge conflicts.
## Summary
If `--config-settings` are provided, we cache the built wheels under one
more subdirectory.
We _don't_ invalidate the actual source (i.e., trigger a re-download) or
metadata, though -- those can be reused even when `--config-settings`
change.
Closes https://github.com/astral-sh/uv/issues/7028.
Let's promote type hints!
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
The generated script now annotates the return type of the dummy function
`hello()`.
## Test Plan
<!-- How was it tested? -->
All existing tests have been synced with this update.
## Summary
This PR adds a more flexible cache invalidation abstraction for uv, and
uses that new abstraction to improve support for dynamic metadata.
Specifically, instead of relying solely on a timestamp, we now pass
around a `CacheInfo` struct which (as of now) contains
`Option<Timestamp>` and `Option<Commit>`. The `CacheInfo` is saved in
`dist-info` as `uv_cache.json`, so we can test already-installed
distributions for cache validity (along with testing _cached_
distributions for cache validity).
Beyond the defaults (`pyproject.toml`, `setup.py`, and `setup.cfg`
changes), users can also specify additional cache keys, and it's easy
for us to extend support in the future. Right now, cache keys can either
be instructions to include the current commit (for `setuptools_scm` and
similar) or file paths (for `hatch-requirements-txt` and similar):
```toml
[tool.uv]
cache-keys = [{ file = "requirements.txt" }, { git = true }]
```
This change should be fully backwards compatible.
Closes https://github.com/astral-sh/uv/issues/6964.
Closes https://github.com/astral-sh/uv/issues/6255.
Closes https://github.com/astral-sh/uv/issues/6860.
## Summary
We now track the discovered `IndexCapabilities` for each `IndexUrl`. If
we learn that an index doesn't support range requests, we avoid doing
any batch prefetching.
Closes https://github.com/astral-sh/uv/issues/7221.
## Summary
We were only applying exclusions when discovering the root, apparently.
Our logic now matches the original intent, which is...
- `exclude` always post-filters `members`.
- We don't treat globs any differently than non-globs.
The one confusing setup that falls out of this is that given:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar"]
```
`foo/bar/baz` **would** be included. To exclude it, you would need:
```toml
members = ["foo/bar/baz"]
exclude = ["foo/bar/*"]
```
Closes https://github.com/astral-sh/uv/issues/7071.
## Summary
If we have a singleton `Range`, we don't need to iterate over the map of
available ranges; instead, we can just get the singleton directly.
Closes#6131.
## Summary
Use a path file (`.pth`) instead of `sitecustomize.py` for configuring
path in emphemeral virtualenvs, overlaying the ephemeral venv on top of
the base `.venv`.
`sitecustomize.py` is a module in the python installation and as such a
unique resource - homebrew pythons on macos already install such a file
and thus uv's `sitecustomize.py`, placed in the ephemeral env, did not
have any effect.
I don't find any documentation explicitly saying that addsitedir is
valid in `.pth` files but from trial it seems to be - and there is the
precedent of the existing _virtualenv.pth _virtualenv.py pair that do
nontrivial operations.
## Test Plan
- Testing on ephemeral venv, resolving to base venv including editable
install in base: done (py3.7, 3.12)
- Testing on homebrew python/macos: done (py3.11)
- tests: run_editable
Fixes#7152
## Summary
Fixes#7081
Treats source distribution `.tgz` the same as `.tar.gz` plans
## Test Plan
Quick Version
```bash
cd $(mktemp -d)
uv init
uv add --dev build
.venv/bin/python -m build -s .
mv -v dist/*tar.gz dist/"$(basename dist/*.tar.gz .tar.gz)".tgz
uv pip install dist/*.tgz
```
Can add a proper test to the branch if requested
Change the registry Python sorting implementation to be easier to
follow, making it clearer what it does and that it is a total order. No
functional changes.
## Summary
Explicitly list the formats and extensions that uv supports, based on
[this
list](86ee8d2c01/crates/distribution-filename/src/extension.rs (L70-L77)).
Not a huge fan of adding the section in `concepts/resolution.md`, but I
did not find a better place. Alternatively we could maybe add a
dedicated page that shortly explains Python package types (wheels,
sdists), where such a section could live?
## Test Plan
Local run of the documentation.
This finally gets rid of our hack for working around "hidden"
state. We no longer do a roundtrip marker serialization and
deserialization just to avoid the hidden state.
This adds new routines to `MarkerTree` for "simplifying" and
"complexifying" a tree with respect to lower and upper Python version
bounds.
In effect, "simplifying" a marker is what you do when you write it to a
lock file. Namely, since `uv.lock` includes a `requires-python` bound at
the top, one can say that it acts as a bound on the supported Python
versions. That is, it establishes a context in which one can assume that
bound is true. Therefore, the markers we write can be simplified using
this assumption.
The reverse is "complexifying" a marker, and it's what you do when you
read a marker from the lock file. Namely, once a marker is read, it can
be very difficult in code to keep the corresponding requires-python
context from the lock file. If you lose track of it and decide to
operate on the "simplified" marker, then it's trivial for that to
produce an incorrect result.
I split this change into its own commit because I'm hoping it
crystalizes what it means when we say "a `MarkerTree` has hidden state."
That is, it isn't so much that there is some explicit member of a
`MarkerTree` that is omitted, but rather, the lower and upper version
bounds on `python_full_version` are are rewritten as "unbounded" when
traversing the ADD for display.
We will actually retain this functionality, but rejigger it so that it's
explicit when we do this. In particular, this simplification has been
problematic for us because it fundamentally changes the truth tables of
a marker expression *unless* you are extremely careful to interpret it
only under the original context in which it was simplified. This is
quite difficult to do generally, and in prior work in #6268, we
completed a refactor where we worked around this type of simplification
and moved it to the edges of uv.
In subsequent commits, we'll re-implement this form of simplification as
a more explicit step.
## Summary
I think a better tradeoff here is to skip fetching metadata, even though
we can't validate the extras.
It will help with situations like
https://github.com/astral-sh/uv/issues/5073#issuecomment-2334235588 in
which, otherwise, we have to download the wheels twice.
(This is part of #5711)
## Summary
@BurntSushi and I spotted that the `derivative` crate is only used for
one enum in the entire codebase — however, it's a proc macro, and we pay
for the cost of (re)compiling it in many different contexts.
This replaces it with a private `Inner` core which uses the regular std
derive macros — inlining and optimizations should make this equivalent
to the other implementation, and not too hard to maintain hopefully
(versus a manual impl of `PartialEq` and `Hash` which have to be kept in
sync.)
## Test Plan
Trust CI?
This PR revives #6129, but is less bold:
* It doesn't rename anything. (I think the rename is probably right
though.)
* It doesn't change the _default_ `Debug` impl. Instead, it offers this
as a new `MarkerTree::debug_graph` method.
I found this pretty useful for debugging since it gives a display format
that is more faithful to the internal representation of a `MarkerTree`.
So I think it's worth having around. But making it available in `Debug`
is perhaps a bridge too far since it isn't as familiar as the typical
PEP 508 representation and isn't as succinct.
I did consider printing this when using `{:#?}` (i.e., the "alternate"
debug representation), but too many things use that (like `insta` I
think) to make it practical.
Closes#6129
## Summary
This has bothered me for a while and should be fairly impactful for
users. It requires a weird implementation, since the
distribution-building crate depends on the cache, and so the prune
operation can't live in the cache, since it needs to access internals of
the distribution-building crate.
Closes https://github.com/astral-sh/uv/issues/7096.
## Summary
Like `uv sync`, you can omit the current project (`--no-emit-project`),
a specific package (`--no-emit-package`), or the entire workspace
(`--no-emit-workspace`).
Closes https://github.com/astral-sh/uv/issues/6960.
Closes#6995.
Follow-up to #6959 and #6961: Use the reachability computation instead
of `propagate_markers` everywhere.
With `marker_reachability`, we have a function that computes for each
node the markers under which it is (`requirements.txt`, no markers
provided on installation) or can be (`uv.lock`, depending on the markers
provided on installation) included in the installation. Put differently:
If the marker computed by `marker_reachability` is not fulfilled for the
current platform, the package is never required on the current platform.
We compute the markers for each package in the graph, this includes the
virtual extra packages and the base packages. Since we know that each
virtual extra package depends on its base package (`foo[bar]` implied
`foo`), we only retain the base package marker in the `requirements.txt`
graph.
In #6959/#6961 we were only using it for pruning packages in `uv.lock`,
now we're also using it for the markers in `requirements.txt`.
I think this closes#4645, CC @bluss.
## Summary
We need to prioritize hashes for the distribution over hashes for the
related packages.
I think this needs to be redone entirely though. I can see other issues
with the current approach.
Closes https://github.com/astral-sh/uv/issues/7059.
## Summary
With #6917, there are a lot more PyPy downloads in `uv python list
--all-versions`. I find it clearer to have all the CPython downloads
listed, then all the PyPy downloads, rather than interspersing them. But
this is subjective, feel free to push back!
## Summary
This PR adds `--package` support to `uv build`, such that you can use
`--package` from anywhere in a workspace to build any member.
If a source directory is provided, we use that as the workspace root.
If a file is provided, we error.
For now, `uv build` only builds the current package, making it
semantically identical to `uv sync`.
## Summary
This PR allows users to run `uv build --wheel ./path/to/source.tar.gz`
to build a wheel from a source distribution. This is also the default
behavior if you run `uv build ./path/to/source.tar.gz`. If you pass
`--sdist`, we error.
## Summary
This PR exposes uv's PEP 517 implementation via a `uv build` frontend,
such that you can use `uv build` to build source and binary
distributions (i.e., wheels and sdists) from a given directory.
There are some TODOs that I'll tackle in separate PRs:
- [x] Support building a wheel from a source distribution (rather than
from source) (#6898)
- [x] Stream the build output (#6912)
Closes https://github.com/astral-sh/uv/issues/1510
Closes https://github.com/astral-sh/uv/issues/1663.
In the `lock_redact_https` test specifically, it prompts a link mode
warning from `uv` on my system. Debugging seems to suggest it is
provoked by attempting to hardlink between `/tmp` and `~/.local`. Since
these are on different file systems for me (with `/tmp` being a
ramdisk), it provokes the warning, and this turn spoils the snapshot
when running tests locally.
This PR adds a test specific filter rule to fix this.
## Summary
The error handlers now happen one level higher, matching on _any_ `Err`
that's returned from the lock-and-sync operations.
Closes https://github.com/astral-sh/uv/issues/7011.
`_virtualenv.py` doesn't need to import `__future__.annotations`, as it
has none.
Removing the import:
* Restores the action of the VIRTUALENV_PATCH on Python 3.6
* Eliminates 24 lines of error messages displayed by Python 3.6 when it
starts in an environment created by uv:
```plaintext
Error processing line 1 of /tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.pth:
Traceback (most recent call last):
File "~/.pyenv/versions/3.6.15/lib/python3.6/site.py", line 168, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "/tmp/tmp.ENwqZ0oeyb/lib/python3.6/site-packages/_virtualenv.py", line 3
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
Remainder of file ignored
```
(Python displays the errors above twice.)
I appreciate the Python team no longer support Python 3.6, but
RedHat-style Linux distributions will support Python 3.6 in their
`/usr/libexec/platform-python` until [releasever 8 expires in
2029](https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide).
I'm happy for the community to move on, in general, but don't see the
harm in helping those who can't.
I'm not yet sure what in the “remainder of file ignored” is necessary
for my project's build, as I haven't yet finished digging that from
under Hatch. I'll follow up on #6426 when I do, so we can concentrate on
getting to the happy cow.
## Test Plan
```sh
( set -eu
export VIRTUAL_ENV="$(mktemp -d)"
./target/release/uv venv "$VIRTUAL_ENV" --python=python3.6
./target/release/uv pip install cowsay
$VIRTUAL_ENV/bin/python -m cowsay --text 'Look, a talking cow!' )
```
Happy output:
```plaintext
Using Python 3.6.15 interpreter at: ~/.local/bin/python3.6
Creating virtualenv at: /tmp/tmp.VHl4XNi3oI
Activate with: source /tmp//tmp.VHl4XNi3oI/bin/activate
Resolved 1 package in 929ms
Installed 1 package in 17ms
+ cowsay==6.0
____________________
| Look, a talking cow! |
====================
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Resolves issues mentioned in comments
* https://github.com/astral-sh/uv/issues/6699#issuecomment-2322515962
* https://github.com/astral-sh/uv/issues/6866#issuecomment-2322785906
Further investigation on the comments revealed that the pointer
arithmethic being performed in `let handle_start = unsafe {
crt_magic.offset(1 + handle_count) };` from [posy
trampoline](dda22e6f90/src/trampolines/windows-trampolines/posy-trampoline/src/bounce.rs (L146))
had some slight errors. Since `crt_magic` was a `*const u32`, doing an
offset by `1 + handle_count` would offset by too much, with some
possible out of bounds reads or attempts to call CloseHandle on garbage.
We needed to offset differently since we want to offset by
`handle_count` bytes after the initial offset as seen in
[launcher.c](888c48b568/PC/launcher.c (L578)).
Similarly, we needed to skip the first 3 handles, otherwise we'd still
be attempting to close standard I/O handles of the parent (in this case
the shell from `busybox.exe sh -l`).
I also added a few extra checks available from `launcher.c` which checks
if the handle value is `-2` just to match the distlib implementation
more closely and minimize differences.
## Test Plan
Manually compiled distlib's launcher with additional logging and
replaced `Lib/site-packages/pip/_vendor/distlib/t64.exe` with the
compiled one to log pointers. As a result, I was able to verify the
retrieved handle memory addresses in this function actually match in
both uv and distlib's implementation from within busybox.exe nested
shell where this behavior can be observed and manually tested.
I was also able to confirm this fixes the issues mentioned in the
comments, at least with busybox's shell, but I assume this would fix the
case with cmake.
## Open areas
`launcher.c` also [checks the
size](888c48b568/PC/launcher.c (L573-L576))
of `cbReserved2` before retrieving `handle_start` which this function
currently doesn't do. If we wanted to, we could add the additional check
here as well, but I wasn't fully sure why it wasn't added in the first
place. Thoughts?
```rust
// Verify the buffer is large enough
if si.cbReserved2 < (size_of::<u32>() as isize + handle_count + size_of::<HANDLE>() as isize * handle_count) as u16 {
return;
}
```
---------
Co-authored-by: konstin <konstin@mailbox.org>
When a package is included under a platform-specific marker, we know
that wheels that mismatch this marker can never be installed, so we drop
them from the lockfile.
In transformers, we have:
* `tensorflow-text`: `tensorflow-macos; python_full_version >= '3.13'
and platform_machine == 'arm64' and platform_system == 'Darwin'`
* `tensorflow-macos`: `tensorflow-cpu-aws; (python_full_version < '3.10'
and platform_machine == 'aarch64' and platform_system == 'Linux') or
(python_full_version >= '3.13' and platform_machine == 'aarch64' and
platform_system == 'Linux') or (python_full_version >= '3.13' and
platform_machine == 'arm64' and platform_system == 'Linux')`
* `tensorflow-macos`: `tensorflow-intel; python_full_version >= '3.13'
and platform_system == 'Windows'`
This means that `tensorflow-cpu-aws` and `tensorflow-intel` can never be
installed, and we can drop them from the lockfile.
## Summary
I'm not convinced that the behavior is correct as-implemented. When the
user passes a `--python >=3.8` or we discover a `requires-python` from
the workspace, we're currently writing that request out to
`.python-version`. I would probably rather that we write the resolved
patch version?
Closes https://github.com/astral-sh/uv/issues/6821.
renovate[bot]
Charlie Marsh
konsti
Jp
adisbladis
Jo
Jiahao Yuan
Zanie Blue
Ahmed Ilyas
Kemal Akkoyun
Aarni Koskela
Liam
tfsingh
Topher Anderson
Andrew Gallant
Bas Schoenmaeckers
Huang, Hong-Chang
Abdó Roig-Maranges
Luca Bruno
bluss
You Jiacheng
kyoto7250
Mikko Leppänen
Aditya Pratap Singh
Mathieu Kniewallner
Dan Watson
Hanlu
Nathan McDougall
Frost Ming
Michał Górny
Ibraheem Ahmed
Bartosz Sławecki
Soof Golan
Shantanu
Janosh Riebesell
Amos Wenger
Alex Gaynor
eth3lbert
Garth Kidd
samypr100