## Summary
A lot of the existing tests were no-ops. For convenience, we now use the
trick of: install from Test PyPI (to get an outdated "latest"), then
upgrade from PyPI.
Right now, the URL gets out-of-sync with the install path, since the
install path is canonicalized. This leads to a subtle error on Windows
(in CI) in which we don't preserve caching across resolution and
installation.
Surprisingly, this is a lockfile schema change: We can't store relative
paths in urls, so we have to store a `filename` entry instead of the
whole url.
Fixes#4355
## Summary
We now persist the `ResolverInstallerOptions` when writing out a tool
receipt. When upgrading, we grab the saved options, and merge with the
command-line arguments and user-level filesystem settings (CLI > receipt
> filesystem).
The loose consensus is that "fetch" doesn't have much meaning and that a
boolean flag makes more sense from the command line.
1. Adds `--allow-python-downloads` (hidden, default) and
`--no-python-downloads` to the CLI to quickly enable or disable
downloads
2. Deprecates `--python-fetch` in favor of the options from (1)
3. Removes `python-fetch` in favor of a `python-downloads` setting
5. Adds a `never` variant to the enum, allowing even explicit installs
to be disabled via the configuration file
## Test plan
I tested this with various `pyproject.toml`-level settings and `uv venv
--preview --python 3.12.2` and `uv python install 3.12.2` with and
without the new CLI flags.
Warn when there are missing bounds on transitive dependencies with
`--resolution lowest`.
Implemented as a lazy resolution graph check. Dev deps are odd because
they are missing the edge from the root that extras have (they are
currently orphans in the resolution graph), but this is more complex to
solve properly because we can put dev dep information in a `Requirement`
so i special cased them here.
Closes#2797
Should help with #1718
---------
Co-authored-by: Ibraheem Ahmed <ibraheem@ibraheem.ca>
## Summary
This PR rewrites the `MarkerTree` type to use algebraic decision
diagrams (ADD). This has many benefits:
- The diagram is canonical for a given marker function. It is impossible
to create two functionally equivalent marker trees that don't refer to
the same underlying ADD. This also means that any trivially true or
unsatisfiable markers are represented by the same constants.
- The diagram can handle complex operations (conjunction/disjunction) in
polynomial time, as well as constant-time negation.
- The diagram can be converted to a simplified DNF form for user-facing
output.
The new representation gives us a lot more confidence in our marker
operations and simplification, which is proving to be very important
(see https://github.com/astral-sh/uv/pull/5733 and
https://github.com/astral-sh/uv/pull/5163).
Unfortunately, it is not easy to split this PR into multiple commits
because it is a large rewrite of the `marker` module. I'd suggest
reading through the `marker/algebra.rs`, `marker/simplify.rs`, and
`marker/tree.rs` files for the new implementation, as well as the
updated snapshots to verify how the new simplification rules work in
practice. However, a few other things were changed:
- [We now use release-only comparisons for `python_full_version`, where
we previously only did for
`python_version`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/algebra.rs#L522).
I'm unsure how marker operations should work in the presence of
pre-release versions if we decide that this is incorrect.
- [Meaningless marker expressions are now
ignored](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/parse.rs#L502).
This means that a marker such as `'x' == 'x'` will always evaluate to
`true` (as if the expression did not exist), whereas we previously
treated this as always `false`. It's negation however, remains `false`.
- [Unsatisfiable markers are written as `python_version <
'0'`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/tree.rs#L1329).
- The `PubGrubSpecifier` type has been moved to the new `uv-pubgrub`
crate, shared by `pep508-rs` and `uv-resolver`. `pep508-rs` also depends
on the `pubgrub` crate for the `Range` type, we probably want to move
`pubgrub::Range` into a separate crate to break this, but I don't think
that should block this PR (cc @konstin).
There is still some remaining work here that I decided to leave for now
for the sake of unblocking some of the related work on the resolver.
- We still use `Option<MarkerTree>` throughout uv, which is unnecessary
now that `MarkerTree::TRUE` is canonical.
- The `MarkerTree` type is now interned globally and can potentially
implement `Copy`. However, it's unclear if we want to add more
information to marker trees that would make it `!Copy`. For example, we
may wish to attach extra and requires-python environment information to
avoid simplifying after construction.
- We don't currently combine `python_full_version` and `python_version`
markers.
- I also have not spent too much time investigating performance and
there is probably some low-hanging fruit. Many of the test cases I did
run actually saw large performance improvements due to the markers being
simplified internally, reducing the stress on the old `normalize`
routine, especially for the extremely large markers seen in
`transformers` and other projects.
Resolves https://github.com/astral-sh/uv/issues/5660,
https://github.com/astral-sh/uv/issues/5179.
## Summary
This PR adds a `DistExtension` field to some of our distribution types,
which requires that we validate that the file type is known and
supported when parsing (rather than when attempting to unzip). It
removes a bunch of extension parsing from the code too, in favor of
doing it once upfront.
Closes https://github.com/astral-sh/uv/issues/5858.
## Summary
I think this seems reasonable... Otherwise, we might not go back to PyPI
to revalidate the list of available versions despite the user passing
`--upgrade`.
## Summary
Previously, we wouldn't respect configuration files in directories
_above_ a workspace root. But this is somewhat problematic, because any
`pyproject.toml` will define a workspace root...
Instead, I think we should _start_ the search at the workspace root, but
go above it if necessary.
Closes: #5929.
See: https://github.com/astral-sh/uv/pull/4295.
## Summary
Resolves#5188. Most of the changes involve creating a new function in
`tool/common.rs` to contain the common functionality previously found in
`tool/install.rs`.
## Test Plan
`cargo test`
```console
❯ ./target/debug/uv tool upgrade black
warning: `uv tool upgrade` is experimental and may change without warning.
Resolved 6 packages in 25ms
Uninstalled 1 package in 3ms
Installed 1 package in 19ms
- black==23.1.0
+ black==24.4.2
Installed 2 executables: black, blackd
```
e.g.
```
❯ cargo run -- venv --no-system
Blocking waiting for file lock on build directory
Compiling uv v0.2.34 (/Users/zb/workspace/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.85s
Running `target/debug/uv venv --no-system`
warning: The `--no-system` flag has no effect, a system Python interpreter is always used in `uv venv`
Using Python 3.12.4 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ cargo run -- venv --system
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
Running `target/debug/uv venv --system`
warning: The `--system` flag has no effect, a system Python interpreter is always used in `uv venv`
Using Python 3.12.4 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```
## Summary
This _used_ to be true but we now require fetching metadata for all
distributions even with `--no-deps` since, e.g., we validate that any
declared extras exist.
## Summary
Initially, we showed _all_ resolver and installer output in `uv run` and
`uv tool run`, since it was way too much for workhorse commands. Then,
we moved to showing _no_ output by default, which was way too little --
you had no idea why anything was happening, and commands appeared to
hang.
This PR adds a more nuanced middle-ground. With `--verbose`, we continue
to show everything. But by default, in `uv run` and `uv tool run`...
- During resolution, we show any "Building" and "Build" messages, if you
need to build a source distribution. But we don't show any other output.
(This _could_ be too little for expensive resolutions; we may want to
show a spinner.)
- If there are no changes to be made after resolving, we don't show any
other output.
- If we have to install, we show the progress bars for downloads (which
disappear on completion) followed by a single summary line stating the
number of packages installed.
This feels pretty good, in my limited testing. When everything is built
/ cached, you don't get _any_ additional output. When there's work to
do, you have a sense for what's happening, and we leave you with a
single summary line ("Installed X packages") at the end.
Closes https://github.com/astral-sh/uv/issues/5758.
## Test Plan
Notice that the first `tool run` ends with an install line; the second
shows no additional output:
If you run `uv run` in a package for the first time, we _do_ tell you
that we're building / built it:
But on the second run, there's no output:
If you add a `--with`, we'll show you all the installer progress bars
(which disappear once they're done), and then a single summary line:
Currently, the entry for a package+version+source table is called
`distribution`. That is incorrect, the `sdist` and `wheel` fields inside
of that table are distributions, the table itself is for a package. We
also align ourselves closer with PEP 751.
I went through `lock.rs` and renamed all occurrences of "distribution"
that actually referred to a "package".
This change invalidates all existing lockfiles.
Bikeshedding: Do we call it `package` or `packages`? See also
https://github.com/python/peps/pull/3877
`package` is nice because it looks like a header:
```toml
[[package]]
name = "anyio"
version = "4.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "idna" },
{ name = "sniffio" },
]
sdist = { url = "3970183622d484d08e3285104333d3/anyio-4.3.0.tar.gz", hash = "sha256:f75253795a87df48568485fd18cdd2a3fa5c4f7c5be8e5e36637733fce06fed6", size = 159642 }
wheels = [
{ url = "2f20c40b45242c0b33774da0e2e34f/anyio-4.3.0-py3-none-any.whl", hash = "sha256:048e05d0f6caeed70d731f3db756d35dcc1f35747c8c403364a8332c630441b8", size = 85584 },
]
```
`packages` is nice because the field is not a single entry, but a list.
2/3 for https://github.com/astral-sh/uv/issues/4893
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Whenever we call `resolve`, we immediately call `fetch` after. And in
some cases `resolve` actually calls `fetch` internally. It seems a lot
simpler to just merge these into one method that returns a `Fetch`
(which itself contains the fully-resolved URL).
Closes https://github.com/astral-sh/uv/issues/5876.
There are three options that determine resolver behavior:
* resolution mode
* prerelease mode
* exclude newer
They are different from the other top level options: If they mismatch,
we recreate the resolution. To distinguish them from the rest of the
lockfile, we group them under an `[options]` header.
1/3 for #4893
Following #5869, the documentation has some less-than-helpful
suggestions to use `uv help python` for details — we should link to the
`uv python` section instead.
## Summary
We were dropping the query and fragment in the wrong place, so the URLs
didn't match up after resolving from an existing lockfile.
Closes https://github.com/astral-sh/uv/issues/5851.
## Summary
Very subtle bug. The scenario is as follows:
- We resolve: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git" }`
- The user then changes the request to: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git", rev =
"44d2f4b19d6837ea990c16f494bdf7543d57483d" }`
- When we go to re-lock, we note two facts:
1. The "default branch" resolves to
`44d2f4b19d6837ea990c16f494bdf7543d57483d`.
2. The metadata for `44d2f4b19d6837ea990c16f494bdf7543d57483d` is
(whatever we grab from the lockfile).
- In the resolver, we then ask for the metadata for
`44d2f4b19d6837ea990c16f494bdf7543d57483d`. It's already in the cache,
so we return it; thus, we never add the
`44d2f4b19d6837ea990c16f494bdf7543d57483d` ->
`44d2f4b19d6837ea990c16f494bdf7543d57483d` mapping to the Git resolver,
because we never have to resolve it.
This would apply for any case in which a requested tag or branch was
replaced by its precise SHA. Replacing with a different commit is fine.
It only applied to `tool.uv.sources`, and not PEP 508 URLs, because the
underlying issue is that we aren't consistent about "automatically"
extracting the precise commit from a Git reference.
Closes https://github.com/astral-sh/uv/issues/5860.
## Summary
This is an experimental PR to replace more unsafe calls with more rust
while still trying to keep the binary size small enough. These changes
roughly increase the size of the trampolines to about 40kb~. This is a
alternate PR to https://github.com/astral-sh/uv/pull/5751.
The primary changes here include
* Switch to use rust path components for ease of path management
* Leverage `std::process::exit` for process exit and cleanup
* Use `std::io::Error::last_os_error` for IO Errors to remove
`FormatMessage` complexity
* Use `std::env::current_exe` to get the current executable instead of
`GetModuleFileNameA`
## Test Plan
Added one more existing test case to trampoline tests.
Still need to verify dunce::canonicalize is desired or not on
find_python_exe.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
We need to avoid using incompatible versions for build dependencies that
are also part of the resolved
environment. This is a very subtle issue, but: when locking, we don't
enforce platform
compatibility. So, if we reuse the resolver state to install, and the
install itself has to
preform a resolution (e.g., for the build dependencies of a source
distribution), that
resolution may choose incompatible versions.
The key property here is that there's a shared package between the build
dependencies and the
project dependencies.
Closes https://github.com/astral-sh/uv/issues/5836.
This already rejects `pyproject.toml`... but because the schema
validation is relaxed (we allow unknown fields, and all fields are
optional), a `pyproject.toml` doesn't get properly rejected here.
This PR makes the schema stricter, but in a safe way (by adding the
other `tool.uv` fields, like `workspace`, as any).
Closes#5832.
## Summary
We allow the use of (e.g.) `.whl.metadata` files when `--no-binary` is
enabled, so it makes sense that we'd also also allow wheels to be
downloaded for metadata extraction. So now, we validate `--no-binary` at
install time, rather than metadata-fetch time.
Closes https://github.com/astral-sh/uv/issues/5699.
## Summary
This fixes a bug introduced by
https://github.com/astral-sh/uv/pull/5232. It turns out that the
`universal_disjoint_base_or_local_requirement` test does not actually do
what it was meant to because of the incorrect python requirement. With a
valid python requirement, it fails on `main`. The problem is that we try
to exclude the original base version from the range of allowed versions
to try and prefer local versions. However, in the test, there is a
branch that depends on the non-local version, with no applicable local
in its fork. We should remove this exclusion as prioritization is
handled by the candidate resolver.
I don't think this will save any time in serialization, but it should
save us some deserialization, since we only need to parse URLs for the
packages we use...
## Summary
Okay, I tested this against...
- Our public "private" proxy
- Fury
- AWS CodeArtifact
- Azure Artifacts
It took a long time.
All of them work as expected with this approach: we omit the credentials
from the lockfile, then wire them back up when the index URL is provided
during subsequent operations.
Closes https://github.com/astral-sh/uv/issues/5119.
Part of #4454
e.g.
```
$ uv add --help
Add one or more packages to the project requirements
Usage: uv add [OPTIONS] <REQUIREMENTS>...
Arguments:
<REQUIREMENTS>... The packages to add, as PEP 508 requirements (e.g., `ruff==0.5.0`)
Options:
--dev Add the requirements as development dependencies
--optional <OPTIONAL> Add the requirements to the specified optional dependency group
--no-editable Don't add the requirements as editables
--raw-sources Add source requirements to `project.dependencies`, rather than `tool.uv.sources`
--rev <REV> Specific commit to use when adding from Git
--tag <TAG> Tag to use when adding from git
--branch <BRANCH> Branch to use when adding from git
--extra <EXTRA> Extras to activate for the dependency; may be provided more than once
--locked Assert that the `uv.lock` will remain unchanged
--frozen Add the requirements without updating the `uv.lock` file
--package <PACKAGE> Add the dependency to a specific package in the workspace
-p, --python <PYTHON> The Python interpreter into which packages should be installed. [env: UV_PYTHON=]
Index options:
-i, --index-url <INDEX_URL> The URL of the Python package index (by default: <https://pypi.org/simple>) [env: UV_INDEX_URL=]
--extra-index-url <EXTRA_INDEX_URL> Extra URLs of package indexes to use, in addition to `--index-url` [env: UV_EXTRA_INDEX_URL=]
-f, --find-links <FIND_LINKS> Locations to search for candidate distributions, in addition to those found in the registry indexes
--no-index Ignore the registry index (e.g., PyPI), instead relying on direct URL dependencies and those provided via `--find-links`
--index-strategy <INDEX_STRATEGY> The strategy to use when resolving against multiple index URLs [env: UV_INDEX_STRATEGY=] [possible values: first-index, unsafe-first-match, unsafe-best-match]
--keyring-provider <KEYRING_PROVIDER> Attempt to use `keyring` for authentication for index URLs [env: UV_KEYRING_PROVIDER=] [possible values: disabled, subprocess]
Resolver options:
-U, --upgrade Allow package upgrades, ignoring pinned versions in any existing output file
-P, --upgrade-package <UPGRADE_PACKAGE> Allow upgrades for a specific package, ignoring pinned versions in any existing output file
--resolution <RESOLUTION> The strategy to use when selecting between the different compatible versions for a given package requirement [env: UV_RESOLUTION=] [possible values: highest, lowest, lowest-direct]
--prerelease <PRERELEASE> The strategy to use when considering pre-release versions [env: UV_PRERELEASE=] [possible values: disallow, allow, if-necessary, explicit, if-necessary-or-explicit]
--exclude-newer <EXCLUDE_NEWER> Limit candidate packages to those that were uploaded prior to the given date [env: UV_EXCLUDE_NEWER=]
Installer options:
--reinstall Reinstall all packages, regardless of whether they're already installed. Implies `--refresh`
--reinstall-package <REINSTALL_PACKAGE> Reinstall a specific package, regardless of whether it's already installed. Implies `--refresh-package`
--link-mode <LINK_MODE> The method to use when installing packages from the global cache [env: UV_LINK_MODE=] [possible values: clone, copy, hardlink, symlink]
--compile-bytecode Compile Python files to bytecode after installation
Build options:
-C, --config-setting <CONFIG_SETTING> Settings to pass to the PEP 517 build backend, specified as `KEY=VALUE` pairs
--no-build Don't build source distributions
--no-build-package <NO_BUILD_PACKAGE> Don't build source distributions for a specific package
--no-binary Don't install pre-built wheels
--no-binary-package <NO_BINARY_PACKAGE> Don't install pre-built wheels for a specific package
Cache options:
-n, --no-cache Avoid reading from or writing to the cache, instead using a temporary directory for the duration of the operation [env: UV_NO_CACHE=]
--cache-dir <CACHE_DIR> Path to the cache directory [env: UV_CACHE_DIR=]
--refresh Refresh all cached data
--refresh-package <REFRESH_PACKAGE> Refresh cached data for a specific package
Python options:
--python-preference <PYTHON_PREFERENCE> Whether to prefer using Python installations that are already present on the system, or those that are downloaded and installed by uv [possible values: only-managed, managed, system, only-system]
--python-fetch <PYTHON_FETCH> Whether to automatically download Python when required [possible values: automatic, manual]
Global options:
-q, --quiet Do not print any output
-v, --verbose... Use verbose output
--color <COLOR_CHOICE> Control colors in output [default: auto] [possible values: auto, always, never]
--native-tls Whether to load TLS certificates from the platform's native certificate store [env: UV_NATIVE_TLS=]
--offline Disable network access, relying only on locally cached data and locally available files
--no-progress Hides all progress outputs when set
--config-file <CONFIG_FILE> The path to a `uv.toml` file to use for configuration [env: UV_CONFIG_FILE=]
--no-config Avoid discovering configuration files (`pyproject.toml`, `uv.toml`) in the current directory, parent directories, or user configuration directories [env: UV_NO_CONFIG=]
-h, --help Print help
-V, --version Print version
Use `uv help add` for more details.
```
## Summary
`uv tree` will now filter to the current platform by default. You can
pass `--universal` to show the entire tree.
Closes https://github.com/astral-sh/uv/issues/5760.
## Summary
It's fine for this to be in the cache, I think, since we don't
necessarily need to colocate it with the Python directory.
Closes https://github.com/astral-sh/uv/issues/5747.
## Summary
After referring to https://github.com/astral-sh/uv/pull/5637 and doing
additional testing.
The default value in a stable state seems more reasonable to be
``only-system``. ``managed`` in preview.
```
cpython-3.11.9-windows-x86_64-none C:\Users\name\AppData\Local\Programs\Python\Python311\python.exe
cpython-3.10.14-windows-x86_64-none C:\Users\name\AppData\Roaming\uv\data\python\cpython-3.10.14-windows-x86_64-none\install\python.exe
cpython-3.10.11-windows-x86_64-none C:\Users\name\AppData\Local\Programs\Python\Python310\python.exe
cpython-3.9.19-windows-x86_64-none C:\Users\name\AppData\Roaming\uv\data\python\cpython-3.9.19-windows-x86_64-none\python.exe
```
test on uv 0.2.33 (build from
257007ccaf)
### Stable version
``uv venv -p 3.10`` is ``3.10.11`` (System Python)
``uv venv -p 3.9`` is ``No interpreter found``(3.9.19 for managed
Python)
``uv venv -p 3.9 --python-preference only-system`` is ``No interpreter
found``(fail)
``uv venv -p 3.9 --python-preference only-managed`` is
``3.9.19``(success)
Do not use managed Python, only use the system Python, so it can be
determined as ``only-system``.
### Preview mode
**Note:** ``3.10.14`` is managed python, ``3.10.11`` is system python.
``uv venv -p 3.11 --preview`` is ``3.11.9`` (System Python)
``uv venv -p 3.10 --preview`` is ``3.10.14``
``uv venv -p 3.10 --preview --python-preference only-managed`` is
``3.10.14``
``uv venv -p 3.10 --preview --python-preference managed`` is ``3.10.14``
``uv venv -p 3.10 --preview --python-preference system`` is ``3.10.11``
``venv -p 3.10 --preview --python-preference only-system`` is
``3.10.11``
Prioritize the managed Python and then select the system Python, so it
can be determined as ``managed``.
-----
fixed#5754
## Test Plan
Run website in local.
## Summary
Right now, if you have a `requirements.txt` with a pre-release, but the
`requirements.in` does not have a pre-release marker for that dependency
we drop the pre-release. (In the selector, we end up returning
`AllowPrerelease::IfNecessary`, the default.)
I played with a few ways of solving this... The first was to remove that
guard altogether. But if we do that,
`universal_transitive_disjoint_prerelease_requirement` fails (we use
`1.17.0rc1` in both forks, when it should only apply to one of the two).
The second was to do that, but also avoid pushing pre-releases as
preferences when we solve a fork. But then
`universal_disjoint_prereleases` fails, because we return a different
pre-release in each fork.
Finally, I settled on allowing existing pre-releases in forks if they
have no markers on them, i.e., they are "global" preferences. I believe
this is true IFF the preference came from an existing lockfile.
Closes https://github.com/astral-sh/uv/issues/5729.
## Summary
If _both_ nodes in the derivation tree are proxies, we need to remove
the _entire_ node. So, the function now returns an `Option<Tree>`
instead of taking `&mut Tree`.
Closes https://github.com/astral-sh/uv/issues/5618.
To enforce the 100 character line limit in markdown files introduced in
https://github.com/astral-sh/uv/pull/5635, and to automate the
formatting of markdown files, i've added prettier and formatted our
markdown files with it.
I've excluded the changelog and the generated references documentation
from this for having too many changes, but we can also include them.
I'm not particular on which style we use. My main motivations are
(major) not having to reflow markdown files myself anymore and (minor)
consistence between all markdown files. I've chosen prettier for similar
reason as we chose black, it's a single good style that's automated and
shared in the community. I do prefer prettier's style of not breaking
inside of a link name though.
This PR is in two parts, the first adds prettier to CI and documents
using it, while the second actually formats the docs. When merge
conflicts arise, we can drop the last commit and regenerate it with `npx
prettier --prose-wrap always --write BENCHMARKS.md CONTRIBUTING.md
README.md STYLE.md docs/*.md docs/concepts/**/*.md docs/guides/**/*.md
docs/pip/**/*.md`.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Partially resolves#5561. Haven't added overrides support yet but I can
add it tomorrow if the current approach for constraints is ok.
## Test Plan
`cargo test`
Manually checked trace logs after changing the constraints.
## Summary
Gives you a nice error message if you attempt to sync with, e.g., `-p
3.8` when that version is supported by at least one workspace member,
but your project's minimum requirement is `>=3.12`
Closes https://github.com/astral-sh/uv/issues/5662.
## Summary
I think it's reasonable to only sync the affected group, e.g., `uv add`
on its own should not require syncing all extras.
Closes https://github.com/astral-sh/uv/issues/4418.
Part of #4454
e.g. for `uv help pip compile`
```
Python options:
--python <PYTHON>
The Python interpreter against which to compile the requirements.
By default, uv uses the virtual environment in the current working directory or any parent
directory, falling back to searching for a Python executable in `PATH`. The `--python`
option allows you to specify a different interpreter.
Supported formats:
- `3.10` looks for an installed Python 3.10 using `py --list-paths` on Windows, or
`python3.10` on Linux and macOS.
- `python3.10` or `python.exe` looks for a binary with the given name in `PATH`.
- `/home/ferris/.local/bin/python3.10` uses the exact Python at the given path.
-p, --python-version <PYTHON_VERSION>
The minimum Python version that should be supported by the resolved requirements (e.g., `3.8` or `3.8.17`).
If a patch version is omitted, the minimum patch version is assumed. For example, `3.8` is mapped to `3.8.0`.
--python-preference <PYTHON_PREFERENCE>
Whether to prefer using Python installations that are already present on the system, or those that are downloaded and installed by uv
Possible values:
- only-managed: Only use managed Python installations; never use system Python installations
- managed: Prefer managed Python installations over system Python installations
- system: Prefer system Python installations over managed Python installations
- only-system: Only use system Python installations; never use managed Python installations
--python-fetch <PYTHON_FETCH>
Whether to automatically download Python when required
Possible values:
- automatic: Automatically fetch managed Python installations when needed
- manual: Do not automatically fetch managed Python installations; require explicit installation
```
## Summary
In #5494, I made breaking changes to the tool receipt format. This would
break existing tools for all users. This PR makes the change
backwards-compatible by supporting deserialization for the deprecated
format.
Closes https://github.com/astral-sh/uv/issues/5680.
## Test Plan
Beyond the automated tests, you can run `cargo run tool list` on your
existing machine.
Before:
```
warning: `uv tool list` is experimental and may change without warning
warning: Ignoring malformed tool `black` (run `uv tool uninstall black` to remove)
warning: Ignoring malformed tool `poetry` (run `uv tool uninstall poetry` to remove)
warning: Ignoring malformed tool `ruff` (run `uv tool uninstall ruff` to remove)
```
After:
```
warning: `uv tool list` is experimental and may change without warning
black v0.1.0
- black
poetry v1.8.3
- poetry
ruff v0.0.60
- ruff
```
## Summary
When we add a new optional group in `uv add`, we never to update the
`pyproject.toml` before locking. Otherwise, we use the stale
`pyproject.toml` and omit the optional group.
Closes https://github.com/astral-sh/uv/issues/5687.
## Summary
Fixes a bug in #5494. The `RequirementSourceWire` representation was
ambiguous, and so the order of the fields meant that all variants were
mapped to `Registry` when deserializing. (So the snapshots were right,
but behaviors were wrong.)
## Summary
This could still be made more robust, but it's not critical, since you
can always `--force`. It's good to handle this case, though, since we
have an explicit error for it.
Closes https://github.com/astral-sh/uv/issues/5490.
It transpires that detecting the directory a script was sourced from is
non-trivial across `bash`, `ksh` and `zsh`.
The previous version was a one-liner and supported `bash` and `zsh` but
not `ksh`.
It is possible to keep the one-liner and add `ksh` support, but that is
mutually-exclusive with `zsh`.
Therefore, the only way to square this circle is to add an `if` block. A
silver lining here is that although longer, the script is probably
easier to follow as there is less code-golfing going on.
## Summary
The current receipt doesn't capture quite enough information. For
example, it doesn't differentiate between editable and non-editable
requirements. This PR instead uses the full `Requirement` type. I think
we should use a custom representation like we do in the lockfile, but
I'm just using the default representation to demonstrate the idea.
## Summary
As-is, if you have a workspace with mixed `requires-python`
requirements, resolution will _never_ succeed, since we'll use the union
as the `requires-python` bound (i.e., take the lowest value), and fail
when we see the package that only supports some more narrow range.
This PR modifies the behavior to take the intersection (i.e., the
highest value), so if you have one package that supports Python 3.12 and
later, and another that supports Python 3.8 and later, we lock for
Python 3.12. If you try to sync or run with Python 3.8, we raise an
error, since the lockfile will be incompatible with that request.
Konsti has a write-up in https://github.com/astral-sh/uv/issues/5594
that outlines what could be a longer-term strategy.
Closes https://github.com/astral-sh/uv/issues/5578.
By resolving for each fork from the lockfile individually and by adding
using preferences for the current fork, we solve the instability #5180.
I've tested the locally and will add the packse test scenarios upstack.
Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198
The comment in the code explains the bulk of this:
```rust
// We previously computed this heuristic freshness lifetime by
// looking at the difference between the last modified header and
// the response's date header. We then asserted that the cached
// response ought to be "fresh" for 10% of that interval.
//
// It turns out that this can result in very long freshness
// lifetimes[1] that lead to uv caching too aggressively.
//
// Since PyPI sets a max-age of 600 seconds and since we're
// principally just interacting with Python package indices here,
// we just assume a freshness lifetime equal to what PyPI has.
//
// Note though that a better solution here is for the index to
// support proper HTTP caching headers (ideally Cache-Control, but
// Expires also works too, as above).
```
We also remove the `heuristic_percent` field on `CacheConfig`. Since
that's actually part of the cache itself, we bump the simple cache
version.
Finally, we add some more `trace!` calls that should hopefully make
diagnosing issues related to the freshness lifetime a bit easier in the
future.
Fixes#5351
## Summary
Given a fork like:
```
pylint < 3 ; sys_platform == 'darwin'
pylint > 2 ; sys_platform != 'darwin'
```
Solving the top branch will typically yield a solution that also
satisfies the bottom branch, due to maximum version selection (while the
inverse isn't true).
To quote an example from the docs:
```rust
// If there's no difference, prioritize forks with upper bounds. We'd prefer to solve
// `numpy <= 2` before solving `numpy >= 1`, since the resolution produced by the former
// might work for the latter, but the inverse is unlikely to be true due to maximum
// version selection. (Selecting `numpy==2.0.0` would satisfy both forks, but selecting
// the latest `numpy` would not.)
```
Closes https://github.com/astral-sh/uv/issues/4926 for now.
## Summary
First part of: https://github.com/astral-sh/uv/issues/4926. We should
solve forks that _don't_ expand the world of supported versions (e.g.,
`python_version >= '3.11'` enables us to select new packages, since we
narrow the supported version range).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
`uv venv` should support adopting python version specified in
`requires-python` from `pyproject.toml`. This allows customization on
the venv setup when syncing from python project.
Closes https://github.com/astral-sh/uv/issues/5552.
It also serves as a workaround to close
https://github.com/astral-sh/uv/issues/5258.
## Test Plan
<!-- How was it tested? -->
1. Run `uv venv` in folder with `pyroject.toml` specifying
`requries-python = "<3.10"`. Python 3.9 is selected for venv.
2. Change to `requries-python = "<3.11"` and run `uv venv` again. Python
3.10 is selected now.
3. Switch to a folder without `pyproject.toml` then run `uv venv`.
Python 3.12 is selected now.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Collapses the previous default into "managed" and makes the "managed"
behavior match "installed". People should use "only-managed" if they
want that behavior, it seems overly complicated otherwise.
## Summary
This PR deprecates the `--isolated` flag. The treatment varies across
the APIs:
- For non-preview APIs, we warn but treat it as equivalent to
`--no-config`.
- For preview APIs, we warn and ignore it, with two exceptions...
- For `tool run` and `run` specifically, we don't even warn, because we
can't differentiate the command-specific `--isolated` from the global
`--isolated`.
## Summary
The culmination of #4730. We now have `uv run --isolated` which always
uses a fresh environment (but includes the workspace dependencies as
needed). This enables you to test with strict isolation (e.g., `uv run
--isolated -p foo` will ensure that `foo` is unable to import anything
that isn't an actual dependency).
Closes#5430.
## Summary
This PR gets rid of the global `--isolated` flag (which serves a bunch
of independent responsibilities right now) on `uv tool run` in favor of
a dedicated `--isolated` flag, which tells uv to avoid re-using an
existing tool environment for this invocation. We'll add the same thing
to `uv run`, to avoid using the base project environment.
This will become a bit clearer in #5466, when we deprecate the
`--isolated` flag on the preview APIs.
## Summary
The idea here is that we hide all resolver output (the grayed out
resolver messages, plus the list of environment modifications) by
default in `uv run` and `uv tool run`. You can pass `--show-resolution`
to re-enable them.
Closes https://github.com/astral-sh/uv/issues/5458.
## Summary
Right now, `--isolated` is read from `uv run` and `uv init` to avoid
discovering the current workspace (or project). This PR moves that
behavior to a dedicated `--no-workspace` flag for `uv init`, and
`--no-project` for `uv run`. They could use the same flag, but
`--no-project` feels confusing for `uv init`, and `--no-workspace` seems
confusing for `uv run` (especially so once you read the documentation,
where we refer to the thing you're omitting as the project).
Closes https://github.com/astral-sh/uv/issues/5429.
This PR represents a different approach to marker propagation in an
attempt to unblock #4640. In particular, instead of propagating markers
when forks are created, we wait until resolution is complete to
propagate all markers to all dependencies in each fork. This ends up
being both more robust (we should never miss anything) and simpler to
implement because it doesn't require mutating a `PubGrubPackage` (which
was pretty annoying). I think the main downside here is that this can
sometimes add markers where they aren't needed.
This actually winds up making quite a few snapshot changes. I went
through each of them. Some of them look like legitimate bug fixes. Some
of them look like superfluous additions. And some of them look like they
would be removed if we had perfect marker normalization. But I don't
think any of the changes are _wrong_.
## Summary
If we just created an entrypoint script, we can of course set the
permissions (we just created it). However, if we're copying from the
cache, we might _not_ own the file. In that case, if we need to change
the permissions (we shouldn't, since the script is likely already
executable -- we set the permissions when we unzip, but I guess they
could _not_ be properly set in the zip itself), we have to copy it.
Closes https://github.com/astral-sh/uv/issues/5581.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
use windows-sys bindings maintained by microsoft devs. winapi didn't has
any updates for more than 3 years
## Test Plan
cargo test. it failed locally because I don't have Python 3.12 installed
## Summary
uv run --directory <path> means that one doesn't have to change to a
project's directory to run programs from it. It makes it possible to use
projects as if they are tool installations.
To support this, first the code reading .python-version was updated so
that
it can read such markers outside the current directory. Note the minor
change this causes (if I'm right), described in the commit.
## Test Plan
One test has been added.
## --directory
Not sure what the name of the argument should be, but it's following uv
sync's directory for now.
Other alternatives could be "--project". Uv run and uv tool run should
probably find common agreement on this (relevant for project-locked
tools).
I've implemented this same change in Rye, some time ago, and then we
went
with --pyproject `<`path to pyproject.toml file`>`. I think using
pyproject.toml file path and not directory was probably a mistake, an
overgeneralization one doesn't need.
Every packse version update is currently causing a huge diff (the size
of the `lock_scenarios.rs` diff in this PR). By redacting the version
from the snapshots, we will only have the actual change in the diff and
not the redundant version change noise.
The second commit moves all remaining packse url arg values to
`common/mod.rs`, which acts as a single source of truth for the packse
version.
## Summary
The package was being installed as editable, but it wasn't marked as
such in `uv pip list`, as the `direct-url.json` was wrong.
Closes https://github.com/astral-sh/uv/issues/5543.
## Summary
The idea here is similar to what we do for wheels: we create the
`CachedEnvironment` in the `archive-v0` bucket, then symlink it to its
content-addressed location. This ensures that we can always recreate
these environments without concern for whether anyone else is accessing
them.
Part of the challenge here is that we want the virtual environments to
be relocatable, because we're now building them in one location but
persisting them in another. This requires that we write relative (rather
than absolute) paths to scripts and entrypoints. The main risk with
relocatable virtual environments is that the scripts and entrypoints
_themselves_ are not relocatable, because they use a relative shebang.
But that's fine for cached environments, which are never intended to
leave the cache.
Closes https://github.com/astral-sh/uv/issues/5503.
## Summary
Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:
* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
`.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.
The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).
Fixes: #3863
## Test Plan
* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
## Summary
This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.
Part of: https://github.com/astral-sh/uv/issues/5263.
## Test Plan
For example:
```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
## Summary
Closes#2187.
The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.
It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.
## Test Plan
Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`
(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.
## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'
Usage: uv.exe pip list --editable
For more information, try '--help'.
```
Consider the following packse scenario:
```toml
[root]
requires = [
"a>=1.0.0 ; python_version < '3.10'",
"a>=1.1.0 ; python_version >= '3.10'",
"a>=1.2.0 ; python_version >= '3.11'",
]
[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```
On current `main`, this produces a dependency on `a` that looks like
this:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```
But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:
```toml
dependencies = [
{ name = "fork-overlapping-markers-basic-a" },
]
```
That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.
The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.
While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.
For the packse scenario test: https://github.com/astral-sh/packse/pull/206
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.
This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.
In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.
So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.
This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.
This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.
Fixes#5086
Partially addresses #4732
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
## Summary
If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.
This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.
Closes: https://github.com/astral-sh/uv/issues/5440.
## Summary
After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.
This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.
Closes https://github.com/astral-sh/uv/issues/5459.
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
## Summary
It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.
This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.
Closes https://github.com/astral-sh/uv/issues/5424.
## Summary
This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).
Now that we support persistent configuration, users should add:
```toml
[tool.uv]
reinstall = ["foo"]
```
If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).
Closes https://github.com/astral-sh/uv/issues/5420.
Two changes split out from the instability work:
* Break `ResolutionGraph::from_state` into methods before adding new
logic to it.
* `ResolutionGraph`: Convert `NodeKey` type to `PackageRef` struct:
Another small refactoring to make subsequent changes easier.
No functional changes.
@BurntSushi I hope this doesn't interfere with your work too much, the
`PackageRef` should at least make debugging panics here easier.
In preparation for the preferences changes with forking, change the
method structure in `CandidateSelector`. Split out into its own PR to
avoid merge conflicts with main. No functional changes.
Consider these requirements from pylint 3.2.5:
```
Requires-Dist: dill >=0.3.6 ; python_version >= "3.11"
Requires-Dist: dill >=0.3.7 ; python_version >= "3.12"
```
We will split on the python version, but then we may pick a version of
`dill` that's `>=0.3.7` in both branches and also have an otherwise
identical resolution in both forks. In this case, we merge both forks
and store only their conjoined markers.
## Summary
Normalize the order of marker expressions on construction. This removes
the distinction between expressions like `os_name == 'Linux'` vs.
`'Linux' == os_name` throughout the codebase. One caveat here is that
the `in` operator does not have a direct inverse, so we introduce
`MarkerOperator::Contains` to handle that case.
I wanted to land this smaller change before some more intrusive changes
as it simplifies the existing code quite a bit.
## Summary
Prior to this change, the resolver would panic if we ran with
`--offline` and `--no-deps` and we had cached metadata for a _package_
(i.e., the versions) but no cached metadata for the _distribution_
(i.e., the specific wheel), since we weren't validating that the
returned metadata in the `--no-deps` case was actually successful. (We
need metadata, even for `--no-deps`, so that we can validate extras.)
## Test Plan
The added test panics on the previous branch.
## Summary
Users can now run `uv cache prune --ci` (open to feedback on the name of
that flag) to remove all pre-built wheels from the cache, leaving behind
zipped, built wheels (which tend to be the most expensive assets to
re-create). This should greatly increase cache performance in CI
environments, since uploading unzipped wheels can actually hurt
performance if you're persisting the uv cache.
Closes https://github.com/astral-sh/uv/issues/5282.
The test asserts that 28 files were removed. But on my system, 27 files
are removed.
This PR is first about debugging what the difference is (since CI
presumably passes with the status quo snapshot). And then I'm thinking
the right way to fix the test failure is with a filter that replaces the
specific number of files removed (limited to what we know to be
correct) with a placeholder.
## Summary
This is surprisingly complex because we need to decide what happens if
you run `uv run` from within a hidden folder, etc. For now, I did the
simplest thing: we just ignore workspace members that are hidden
directories if they lack a `pyproject.toml`, so you can still include
hidden members, they're just ignored if they don't seem to be projects.
Closes https://github.com/astral-sh/uv/issues/5403.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Implements #5340
## Test Plan
<!-- How was it tested? -->
## Summary
This is a bit simpler than #5333, but seems to work in my testing on
macOS and Windows. It's based on implementations that I found in
[Pixi](36f1bb297d/src/cli/exec.rs (L99))
and
[Wasmer](49e60af8df/lib/wasix/src/state/builder.rs (L1058)).
Closes https://github.com/astral-sh/uv/issues/5257.
## Test Plan
On both macOS and Windows:
- `cargo run -- tool run --from jupyterlab jupyter-lab` -- hit Ctrl-C;
verify that the process exits and the terminal is left in a good state.
- `cargo run -- run python` -- hit Ctrl-C; verify that the process does
_not_ exit, but does on Ctrl-D.
## Summary
Closes https://github.com/astral-sh/uv/issues/5359.
## Test Plan
Unfortunately, the only packages I know of that use this are Ruff and
uv, and both are too heavy to install in a recurring test, so:
`uv tool install hatch==1.12.0 --with uv==0.2.27 --force
--link-mode=symlink`
> DEBUG Found `cpython-3.12.1-macos-aarch64-none` at
`/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
Instead of `<implementation> <version>`
> DEBUG Found cpython 3.12.1 at `/Users/zb/Library/Application
Support/uv/python/cpython-3.12.1-macos-aarch64-none/bin/python3`
(managed installations)
## Summary
Prefers, in order:
- The major-minor version of an interpreter discovered via `--python`.
- The `requires-python` from the workspace.
- The major-minor version of the default interpreter.
If the `--python` request is a version or a version range, we use that
without fetching an interpreter.
Closes https://github.com/astral-sh/uv/issues/5299.
The `RequirementsTxtComparator` was written assuming there is one
distribution per package name. This changed with the universal
resolution, which allows multiple versions or urls for the same package
name. The sorting we emitted for these new entries was incidental.
With this change, we properly sort these entries by name, version and
then url in universal mode.
This is an output format change for `--universal` users.
## Summary
This PR avoids an `Invalid package name` error that occurs when using
`uv init .`. This is achieved by slightly reorganizing the code block to
determine the name after the path is canonicalized. The dot path is
expanded to the current directory, and the `file_name` then works as
expected.
Resolve#5329 .
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
1. Fixes errors from https://github.com/astral-sh/uv/pull/4878
2. More cleanup. Removed the need for `MaybeUninit` and `SizeOf`
helpers. Renamed main entrypoint to expected default in windows
`mainCRTStartup` to avoid re-declaring /ENTRY in build.rs.
3. Adds a small basic test harness that >>on windows<< will generate
both types of launchers and run them. I've had been using this locally
to test changes and edge cases, but it might be useful for others. It's
based on core parts of install-wheel-rs.
## Test Plan
Tested locally on a couple of script/gui apps.
---------
Co-authored-by: konsti <konstin@mailbox.org>
## Summary
Similiar to https://github.com/astral-sh/rye/pull/680, I have made a
major refactor to the `fetch-download-metadata.py` script.
Some notable changes:
- Use PEP 723 inline scripts
- Fully type annotated the script
- Implemented async HTTP fetching
- Introduced a `Finder` base class and move finder logic under
`CPythonFinder` subclass, which will make it easier to add a
`PyPyFinder` later.
- Instead of fetching `xxx.sha256` for each file, the script now fetches
a single `SHA256SUMS` file containing checksums for all files in the
release.
As a result, the script now takes around 10 seconds instead of 10+
minutes.
## Plan for Future PRs
- [ ] Implement the `PyPyFinder`
- [ ] Add an GitHub Action to run `fetch-download-metadata.py` daily and
create PR automatically
## Test Plan
```sh
cargo run -- run --isolated -- ./crates/uv-python/fetch-download-metadata.py
```
## Summary
In my setup, I have a directory of wheels symlinked from different
directories. I can point `--find-links` at it with `pip` and it works
but not `uv`.
Currently, `uv` checks if a candidate file `is_file` which is for
regular files. By also checking `is_symlink` I was able to install a
symlinked wheel. I'm not *exactly* sure where, but some other place is
eventually resolving the absolute path of the wheel. (`uv`? The OS?)
## Test Plan
Manually tested - I didn't see any tests for `FlatIndexClient` in the
`uv-client` crate.
```
mkdir /tmp/a /tmp/b # Create a directory of wheels (/tmp/a) and a directory of symlinked wheels (/tmp/b)
cp test-0.0.1-py3-none-any.whl /tmp/a # Add a wheel to the directory of wheels
ln -s /tmp/a/test-0.0.1-py3-none-any.whl /tmp/b/ # Create a symlink to that wheel
uv pip install test --find-links /tmp/b # Install pointing at the symlinked wheels directory
```
## Summary
Otherwise, if the path is already a member, discovery fails.
Also adds a failing test for "adding members that are already covered by
`members`".
## Summary
Excellent find from @konstin. If we have a package that's included in
two forks at the same version, but with different URLs, we need to avoid
collapsing them in the lockfile.
Closes https://github.com/astral-sh/uv/issues/5294.
## Summary
You can still generate instabilities, but at least it's consistent
between including and excluding the extra.
For example, this resolves to 54 and then 52 packages on re-run:
```toml
[project]
name = "transformers"
version = "4.39.0.dev0"
description = "State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow"
requires-python = ">=3.9.0"
dependencies = []
[project.optional-dependencies]
flax = ["jaxlib>=0.4.1,<=0.4.13"]
onnxruntime = ["onnxruntime>=1.4.0"]
ray = ["ray[tune]>=2.7.0"]
deepspeed-testing = [
"dill<0.3.5",
"datasets!=2.5.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
```
I think the difference is just somewhere in PubGrub -- like, we add an
extra dependency, so the iteration order gets changed, and we end up
with a different resolution at the end.
Closes https://github.com/astral-sh/uv/issues/5285.
## Summary
Per https://github.com/astral-sh/uv/pull/5250#issuecomment-2242137762
> It would also be great to have an argument (perhaps leveraging the
global isolated option) that allows us to disable workspace discovery
when we don't want to add a project as a member.
## Test Plan
```sh
$cargo run -- init work
$ cargo run -- init work/pkg-a --isolated
warning: `uv init` is experimental and may change without warning
Initialized project sub-c in /tmp/work
```
Looking at how to merge identical forks, i found that the `Resolution`
can be simplified by treating it as a nodes and edges store (plus pins,
they are separate since they are per name-version, not per
(virtual-)package-version). This should also make #5294 more apparent,
which i didn't touch here.
I additionally added some doc comments to the `Resolution` types.
Our everything-method `solve` tends to grow large, so before i'm adding
more logic, i'm moving some code and some logging statements around to
keep it manageable.
I made minor changes to the logging, otherwise no logic changes, only
refactoring.
## Summary
`dearpygui==1.9.1 has no wheels are available with a matching Python
ABI` is way better than `he requested Python version (>=3.12.3) does not
satisfy Python>=3.12.3`.
Closes https://github.com/astral-sh/uv/issues/5284.
This fixes resolving packages that publish an invalid stub to pypi, such
as tensorrt-llm.
## Summary
In https://github.com/astral-sh/uv/pull/3138 , we implemented
`unsafe-best-match`. However, it seems to not quite work as expected.
When multiple indices contain the same version, it's not clear which
index the current code uses. This PR fixes that to use the first index
the package is in.
## Test Plan
```console
$ echo 'tensorrt-llm==0.11.0' | ./target/debug/uv pip compile - --extra-index-url https://pypi.nvidia.com --python-version=3.10 --index-strategy=unsafe-best-match --annotation-style=line
```
## Summary
Given `Requires-Python: ">=3.12.3"`, we were rejecting wheels like
`dearpygui-1.11.1-cp312-cp312-win_amd64.whl`, since `3.12.0` is not
included in `>=3.12.3`. We instead need to test against the major-minor
version of `Requires-Python`.
The easiest way to do this, I think, is the use the `RequiresPython`
struct, which has a single bound that we can truncate the major-minor.
This also means that we now allow
`dearpygui-1.11.1-cp312-cp312-win_amd64.whl` for specifiers like
`Requires-Python: "==3.10.*"`. This is incorrect on the surface, but it
does match our semantics for `Requires-Python` elsewhere: we treat it as
a lower bound.
Closes https://github.com/astral-sh/uv/issues/5287.
Emulate `pip`'s behaviour and find `pythonw` executable by doing an
`s/python/pythonw/g` style transformation, as opposed to assuming a
constant `pythonw.exe` path.
See #5256 for more detail e.g. why this is a useful behaviour to
emulate.
Fixes: #5256
## Summary
This PR modifies the lockfile to include the impactful resolution
settings, like the resolution and pre-release mode. If any of those
values change, we want to ignore the existing lockfile. Otherwise,
`--resolution lowest-direct` will typically have no effect, which is
really unintuitive.
Closes https://github.com/astral-sh/uv/issues/5226.
## Summary
I don't think that "always reinstall" is tenable for `uv run`. My
perspective on this is that if you want "always reinstall", you can now
set it persistently in your `pyproject.toml` or `uv.toml`.
As a smaller change, we could instead disable this _only_ for the
Project API.
Closes https://github.com/astral-sh/uv/issues/4946.
Resolves#4467.
## Summary
This PR implements the following
1. Add `tool.uv.constraint-dependencies` to pyproject.toml
1. Support to refer `tool.uv.constraint-dependencies` in `uv lock`
1. Support to refer `tool.uv.constraint-dependencies` in `uv pip
compile/install`
These are analogues of the override features implemented in #3839 and
#4369.
## Test Plan
Add test.
## Summary
resolves https://github.com/astral-sh/uv/issues/5217
## Test Plan
existing tests pass (should be perfectly backwards compatible) + added a
few tests to cover the new functionality. in particular, in addition to
the simple use of `--show-version-specifiers`, its interaction with
`--invert` and `--package` flags are tested.
## Summary
This fixes a few bugs introduced by
https://github.com/astral-sh/uv/pull/5104. I previously thought we could
track conflicting locals the same way we track conflicting URLs in
forks, but it turns out that ends up being very tricky. URL forks work
because we prioritize directly URL requirements. We can't prioritize
locals in the same way without conflicting with the URL prioritization
(this may be possible but it's not trivial), so we run into issues where
a correct resolution depends on the order in which dependencies are
traversed.
Instead, we track local versions across all forks in `Locals`. When
applying a local version, we apply all locals with markers that
intersect with the current fork. This way we end up applying some local
versions without creating a fork. For example, given:
```
// pyproject.toml
dependencies = [
"torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
]
// requirements.in
torch==2.0.0
.
```
We choose `2.0.0+cu118` in all cases. However, if a disjoint fork is
created based on local versions, the resolver will choose the most
compatible local when it narrows to a specific fork. Thus we correctly
respect local versions when forking:
```
// pyproject.toml
dependencies = [
"torch==2.0.0+cu118 ; platform_machine == 'x86_64'",
"torch==2.0.0+cpu ; platform_machine != 'x86_64'"
]
// requirements.in
torch==2.0.0
.
```
We should also be able to use a similar strategy for
https://github.com/astral-sh/uv/pull/5150.
## Test Plan
This fixes https://github.com/astral-sh/uv/issues/5220 locally for me,
as well as a few other bugs that were not reported yet.
## Summary
The current code was checking every constraint against every
requirement, regardless of whether they were applicable. In general,
this isn't a big deal, because this method is only used as a fast-path
to skip resolution -- so we just had way more false-negatives than we
should've when constraints were applied. But it's clearly wrong :)
## Test Plan
- `uv venv`
- `uv pip install flask`
- `uv pip install --verbose flask -c constraints.txt` (with `numpy<1.0`)
Prior to this change, Flask was reported as not satisfied.
## Summary
Implements the `uv init` command, which initializes a project
(`pyproject.toml`, `README.md`, `src/__init__.py`) in the current
directory, or in the given path. `uv init` also does workspace
discovery.
Resolves https://github.com/astral-sh/uv/issues/1360.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Spotted some issues in the settings documentation, and room for small
improvements by linking to PEPs/RFCs.
Also updating contribution documentation to mention that it's necessary
to enable the virtual environment before running `mkdocs serve`.
## Test Plan
Running documentation locally.
Fix#5211
## Summary
Change to show only the python installed on the system if
`--python-preference only-system` is specified.
Below is an example of running the command before the change, showing
Python not installed on the system.
#### Before
```bash
# Check system python
$ uv python --preview list --python-preference only-system
cpython-3.12.4-linux-x86_64-gnu <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.11.9-linux-x86_64-gnu <download available>
cpython-3.10.14-linux-x86_64-gnu <download available>
cpython-3.9.19-linux-x86_64-gnu <download available>
cpython-3.8.19-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
```
This PR changes the display to show only Python installed on the system.
#### After
```bash
$ cargo run python --preview list --python-preference only-system
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
Running `target/debug/uv python list --python-preference only-system --preview`
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu /bin/python3
```
## Test Plan
- `cargo run python list --python-preference only-system` in Ubuntu
24.04 to verify the display.
- `cargo run python list` in Ubuntu 24.04 to verify that the results
before and after the change were the same.
## Summary
This ensures that we process Python installs and uninstalls as soon as
they complete, rather than waiting for them all to complete, then
processing them sequentially. In practice, it shouldn't be much of a
difference (since the processing is code is fairly light), but it
strikes me as more correct.
## Summary
Addressing this [issue](https://github.com/astral-sh/uv/issues/5147) by
adding the capability for Symbolic linking as a link mode when
installing or syncing dependencies.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Currently, `uv` refuses to install anything on GraalPy. This is
currently blocking GraalPy testing with cibuildwheel, since manylinux
includes both `uv` and `graalpy` (but doesn't test with `uv`), whereas
cibuildwheel defaults to `uv`. See e.g.
https://github.com/pypa/cibuildwheel/actions/runs/9956369360/job/27506182952?pr=1538
where it gives
```
+ python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
* Creating isolated environment: venv+uv...
* Using external uv from /usr/local/bin/uv
* Installing packages in isolated environment:
- setuptools >= 40.8.0
> /usr/local/bin/uv pip install "setuptools >= 40.8.0"
< error: Unknown implementation: `graalpy`
```
## Test Plan
I simply based the GraalPy support on PyPy and added some small tests.
I'm open to discussing how to test this. GraalPy is available for
manylinux images and with setup-python, so we should be able to add
tests against it to the CI. I locally confirmed by installing `uv` into
a GraalPy venv and then trying things like `uv pip install Pillow` and
testing those extensions.
## Summary
Closes https://github.com/astral-sh/uv/issues/5124.
## Test Plan
Ran `cargo run -- help pip compile` on my Windows machine, which failed
before but succeeds after this change.
## Summary
You can now use `uv run --locked` to assert that the lockfile doesn't
change, or `uv run --frozen` to run without attempting to update the
lockfile at all.
Closes https://github.com/astral-sh/uv/issues/5185.
As user, you specify a list of extras. Internally, we decompose this
into one virtual package per extra. We currently leak this abstraction
by writing one entry per extra to the lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas" },
{ name = "pandas", extra = "excel" },
{ name = "pandas", extra = "hdf5" },
{ name = "pandas", extra = "html", marker = "os_name != 'posix'" },
{ name = "pandas", extra = "output-formatting", marker = "os_name == 'posix'" },
{ name = "pandas", extra = "plot", marker = "os_name == 'posix'" },
]
```
Instead, we should merge the extras into a list of extras, creating a
more concise lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas", extra = ["excel", "hdf5"] },
{ name = "pandas", extra = ["html"], marker = "os_name != 'posix'" },
{ name = "pandas", extra = ["output-formatting", "plot"], marker = "os_name == 'posix'" },
]
```
The base package is now implicitly included, as it is in PEP 508.
Fixes#4888
## Summary
Makes the `tools()` return value include per-tool errors. This makes it
easy to skip (rather than failing) in `uv tool list`, _and_ improves `uv
tool uninstall` to remove those invalid tools, rather than leaving them
around. (We already had that behavior for `uv tool uninstall ruff` with
an invalid `ruff`, but `uv tool uninstall --all` just left them.)
Closes https://github.com/astral-sh/uv/issues/5151.
Search for all `python3.x` minor versions in PATH, skipping those we
already know we can use.
For example, let's say `python` and `python3` are Python 3.10. When a
user requests `>= 3.11`, we still need to find a `python3.12` in PATH.
We do so with a regex matcher.
Fixes#4709
Warn when there is a direct dependency without a lower bound and
`--resolution lowest` is set.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
"Bare" made sense when we had a variant that seeded the environment, but
now that the crate _only_ creates a bare environment, lets drop that
terminology.
## Summary
Hashes will be validated if present, but aren't required (since, e.g.,
some registries will omit them, as will Git dependencies and such).
Closes https://github.com/astral-sh/uv/issues/5168.
## Summary
This is an alternative to `--require-hashes` which will validate a hash
if it's present, but ignore requirements that omit hashes or are absent
from the lockfile entirely.
So, e.g., transitive dependencies that are missing will _not_ error; nor
will dependencies that are included but lack a hash.
Closes https://github.com/astral-sh/uv/issues/3305.
## Summary
We only need to store one hash -- it should be the "strongest" hash. In
practice, most registries (like PyPI) only serve one, and we only
compute a SHA256 hash for direct URLs.
Part of: https://github.com/astral-sh/uv/issues/4924
## Test Plan
I verified that changing:
```diff
diff --git a/crates/distribution-types/src/hash.rs b/crates/distribution-types/src/hash.rs
index 553a74f55..d36c62286 100644
--- a/crates/distribution-types/src/hash.rs
+++ b/crates/distribution-types/src/hash.rs
@@ -31,7 +31,7 @@ impl<'a> HashPolicy<'a> {
pub fn algorithms(&self) -> Vec<HashAlgorithm> {
match self {
Self::None => vec![],
- Self::Generate => vec![HashAlgorithm::Sha256],
+ Self::Generate => vec![HashAlgorithm::Sha256, HashAlgorithm::Sha512],
Self::Validate(hashes) => {
let mut algorithms = hashes.iter().map(HashDigest::algorithm).collect::<Vec<_>>();
algorithms.sort();
```
Then running `uv lock` with a URL gave me:
```toml
[[distribution]]
name = "iniconfig"
version = "2.0.0"
source = { url = "62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl" }
wheels = [
{ url = "62565a6e1ceac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha512:44cc53a6c8dd7cf4d6d52bded308bcc4b4f85fff2ed081f60f7d4beaa86a7cde6d099e3976331232d4cbd472ad5d1781064725b0999c7cd3a2a4d42df687ee81" },
]
```
## Summary
It turns out that if `path` is a symlink,
`File::create(path)?.write_all(content.as_ref())?` will overwrite the
_target_ file. That means an entrypoint named `python` would actually
overwrite the user's source Python executable, which is symlinked into
the virtual environment.
This PR replaces that code with our atomic write method.
Closes https://github.com/astral-sh/uv/issues/5152.
## Test Plan
I ran through the test plan
`https://github.com/astral-sh/uv/issues/5152`, but used an executable
named `bar` linked to `foo.txt` instead...
* Use a dedicated `ResolverMarkers` check in the fork state. This is
better than the `MarkerTree::And(Vec::new())` check.
* Report the timing correct naming universal resolution instead of two
spaces around an empty string when there are no markers.
* On resolution error, show the split that we're in. I'm not sure how to
word this, since we're doing a universal resolution until we fork, so
the trace may contain information from requirements that are not part of
this fork.
## Summary
Resolves#5139
`PythonInstallationKey` was sorted as a string, which caused `3.8` to
appear before `3.11`. This update changes the sorting of
`PythonInstallationKey` to be a descending order by version.
## Test Plan
```sh
$ cargo run -- python install 3.8 3.12
$ cargo run -- tool run -v python -V
DEBUG uv 0.2.25
warning: `uv tool run` is experimental and may change without warning.
DEBUG Searching for Python interpreter in managed installations, system path, or `py` launcher
DEBUG Searching for managed installations at `C:\Users\xx\AppData\Roaming\uv\data\python`
DEBUG Found managed Python `cpython-3.12.3-windows-x86_64-none`
DEBUG Found cpython 3.12.3 at `C:\Users\xx\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none\install\python.exe` (managed installations)
DEBUG Using request timeout of 30s
DEBUG Using request timeout of 30s
DEBUG Acquired lock for `C:\Users\nigel\AppData\Roaming\uv\data\tools`
DEBUG Using existing environment for tool `httpx`: C:\Users\xx\AppData\Roaming\uv\data\tools\httpx
DEBUG Using existing tool `httpx`
DEBUG Running `httpx -v`
```
## Summary
Fixes#5082.
Adds a new `Printer::NoProgress` that is identical to `Printer::Default`
but doesn't draw any progress bar.
## Test Plan
It seems to me that as of now it's not possible to use `insta-cmd` to
get any progress bar in the comparable output of command.
Best way to test this would be to run any command that usually shows
progress indicators like `uv pip install` with and without
`--no-progress` options.
## Summary
First part of https://github.com/astral-sh/uv/issues/5093.
Remaining:
- Global settings
- `pip`-specific settings (some will be copied-over from here)
- Auto-generating the "Possible values" for enums
Add support for path dependencies from a package in one workspace to a
package in another workspace, which it self has workspace dependencies.
Say we have a main workspace with packages `a` and `b`, and a second
workspace with `c` and `d`. We have `a -> b`, `b -> c`, `c -> d`. This
would previously lead to a mangled path for `d`, which is now fixed.
Like distribution paths, we split workspace paths into an absolute
install path and a relative (or absolute, if the user provided an
absolute path) lock path.
Part of https://github.com/astral-sh/uv/issues/3943
Fixes a concurrency issue when multiple processes are installing the
same package in different virtual environments from Git ref (not a
specific Git commit).
## Symptoms
That's how some of symptoms looked like in our case:
```
DEBUG uv 0.2.21
DEBUG Checking for Python interpreter at path `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python`
DEBUG Using Python 3.12.4 environment at /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7/bin/python
DEBUG Acquired lock for `/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/37bf51bfba4699a940ce31349422b24a5bc55a2b179ed7aec74459a9ae8d57b7`
DEBUG At least one requirement is not satisfied: torch
DEBUG Using request timeout of 300s
DEBUG Found 37 packages in `--find-links` entry: /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/pip/wheels
DEBUG Updating git source `Url { scheme: "https", cannot_be_a_base: false, username: "***", password: None, host: Some(Domain("github.com")), port: None, path: "/iterative/datachain", query: None, fragment: None }`
DEBUG Attempting GitHub fast path for: https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test
DEBUG failed to check github HTTP status client error (404 Not Found) for url (https://api.github.com/repos/iterative/datachain/commits/fix-distributed-test)
DEBUG Performing a Git fetch for: https://***@github.com/iterative/datachain
error: Failed to download and build: `datachain @ git+https://***@github.com/iterative/datachain@fix-distributed-test`
Caused by: Git operation failed
Caused by: process didn't exit successfully: `git clone --local /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/db/9d45a3e6f56b0a69 /tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8` (exit status: 128)
--- stderr
fatal: destination path '/tmp/pytest-of-runner/pytest-0/tmp_venv_dir/python3.12/.cache/uv/git-v0/checkouts/9d45a3e6f56b0a69/56b15b8' already exists and is not an empty directory.
```
## Cause of the issue
It is the same command that is failing - `git clone`, and I think it's
happening because it was trying to first get the repo to dereference the
`fix-distributed-test` branch:
`Given a remote source distribution, return a precise variant, if
possible.`
And it's happening w/i acquiring a lock around cache.
## Fix
I thinks we can reuse the existing `fetch` method that has already lock
around cache:
https://github.com/astral-sh/uv/pull/5051/files#diff-f58bb99dee2c4922d156ace3e7de651f0d9a81fc8e9447a2ad865de5c53543fcR61-R68
```python
// Avoid races between different processes, too.
let lock_dir = cache.join("locks");
....
```
## Questions
- Are there any tests that cover concurrency? I'm quite new to Rust and
if someone can point me to some examples and I can create a similar test
or a new one.
- Is error handling done correctly in this PR (again, I'm new to Rust -
I'll review and read about it, but it's better also for someone else to
review this)
## Summary
Currently, the `Locals` type relies on there being a single local
version for a given package. With marker expressions this may not be
true, a similar problem to https://github.com/astral-sh/uv/pull/4435.
This changes the `Locals` type to `ForkLocals`, which tracks locals for
a given fork. Local versions are now tracked on `PubGrubRequirement`
before forking.
Resolves https://github.com/astral-sh/uv/issues/4580.
Specifically, this shows the resolution produced by the
resolver *before* constructing a resolution graph.
Unlike most trace messages, this is a multi-line message
that needs to do some small amount of work to build
itself. So we do an explicit gating on the log level here
instead of just relying on the `trace!` macro itself.
I messed up the order of checks in #5033, due to which it failed to
exclude the case of `-P package`, as `arg.startswith("-P")` check came
first and skipped only the first argument.
That means that, in the following command:
```console
uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in -P attrs==18.1.0
```
The generated header would exclude `-P`, but keep `attrs==18.1.0`.
```plaintext
# This file was autogenerated by uv via the following command:
# uv pip compile --output-file pip_compile_uv_header.txt unpinned_uv.in attrs==18.1.0
```
But we want to check for an exact match first and then only check for
the case when option and value are together.
This also affected `--find-links` short option of style `-f <uri>`.
Hopefully, third times going to be a charm. 😳
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
I tested locally, and also changed one snapshot test to use `-P` for
variation. I don't think it's worth an extra test, but can do that for
sure.
## Summary
When range requests aren't supported, we fall back to streaming the
wheel, stopping as soon as we hit a `METADATA` file. This is a small
optimization, but the downside is that we don't get to cache the
resulting wheel...
We don't know whether `METADATA` will be at the beginning or end of the
wheel, but it _seems_ like a better tradeoff to download and cache the
entire wheel?
Closes: https://github.com/astral-sh/uv/issues/5088.
Sort of a revert of: https://github.com/astral-sh/uv/pull/1792.
## Summary
Workaround the `stream_wheel` not retry issue
[found](https://github.com/astral-sh/uv/issues/3514#issuecomment-2229820667)
in #3514, it's not a perfect solution but I think it's acceptable
because the error should not occur frequently.
## Test Plan
Manually using `iptables -A OUTPUT -p tcp -dport 3128 -j REJECT
--reject-with tcp-reset` to inject connection reset error to the HTTP
proxy that proxies PyPI requests.
```
error: Failed to prepare distributions
Caused by: Failed to fetch wheel: piqp==0.4.1
Caused by: Request failed after 3 retries
Caused by: error sending request for url (09ade94dfdd3c368ac505b6ca09831/piqp-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl)
Caused by: client error (Connect)
Caused by: tcp connect error: Connection refused (os error 111)
Caused by: Connection refused (os error 111)
```
## Summary
`echo "torch==1.10.0" | cargo run pip compile - -p 3.12 --no-deps` now
correctly fails. Previously, we were accepting the wheel
`torch-1.10.0-cp36-none-macosx_10_9_x86_64.whl` as compatible with
Python 3.10 due to the `none` ABI.
Closes https://github.com/astral-sh/uv/issues/5085.
## Summary
The example in the linked issue doesn't quite work, but I think it has
to do with the existing filtering logic. Will follow-up separately.
Closes https://github.com/astral-sh/uv/issues/5012.
## Summary
So this PR introduces change to how `Array` of dependencies
representation is reformatted while `PyProjectTomlMut` is manipulated.
These changes are here for it to respect the original indentation.
Closes https://github.com/astral-sh/uv/issues/5009
## Test Plan
Using `pyproject.toml` like
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests"
]
```
Executed
```
$ uv add httpx
```
And expected in `pyproject.toml`
```
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests",
"httpx",
]
```
Preserving original indentation
This name should lead to less confusion. Unfortunately this is a
"breaking cache change" so everyone's cache will be invalidated. I'm not
sure if we should support a rename-on-upgrade.
edit: We can make the breaking change next time we bump the version
## Summary
Resolves#5017.
Note: This re-uses the same function defined in #5019 to find matching
packages.
## Test Plan
`cargo test`
```console
❯ ./target/debug/uvx --from fastapi fastapi
warning: `uvx` is experimental and may change without warning.
Resolved 33 packages in 427ms
warning: The fastapi executable is not part of the fastapi package. It is provided by the fastapi-cli package. Use `uvx --from fastapi-cli fastapi` instead.
Usage: fastapi [OPTIONS] COMMAND [ARGS]...
Try 'fastapi --help' for help.
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Should fix#2092.
This PR changes `uv venv` so it also creates symlinks to `pypy` on Unix
and copies executables on Windows when creating a new environment using
PyPy.
I found a bit of discrepancy between creation of a venv using `python`
and `uv`, as using `python` brings all the executables with it. While
`uv` brings only those without any version number, at least on Windows.
The behaviour is different on Unix as we take the versioned symlinks
too.
Some examples below.
`python -m venv` generates the following `Scripts` folder.
```
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/14/2024 15:41 2031 activate
-a---- 7/14/2024 15:41 1029 activate.bat
-a---- 7/14/2024 15:41 9033 Activate.ps1
-a---- 7/14/2024 15:41 393 deactivate.bat
-a---- 7/14/2024 15:40 27648 libffi-8.dll
-a---- 7/14/2024 15:41 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 15:41 108424 pip.exe
-a---- 7/14/2024 15:41 108424 pip3.10.exe
-a---- 7/14/2024 15:41 108424 pip3.exe
-a---- 7/14/2024 15:41 79360 pypy.exe
-a---- 7/14/2024 15:41 79360 pypy3.10.exe
-a---- 7/14/2024 15:41 79360 pypy3.10w.exe
-a---- 7/14/2024 15:41 79360 pypy3.exe
-a---- 7/14/2024 15:41 79360 pypyw.exe
-a---- 7/14/2024 15:41 79360 python.exe
-a---- 7/14/2024 15:41 79360 python3.10.exe
-a---- 7/14/2024 15:41 79360 python3.exe
-a---- 7/14/2024 15:41 79360 pythonw.exe
```
`uv venv` instead generates this.
```
-a---- 7/14/2024 16:27 3360 activate
-a---- 7/14/2024 16:27 2251 activate.bat
-a---- 7/14/2024 16:27 2627 activate.csh
-a---- 7/14/2024 16:27 4191 activate.fish
-a---- 7/14/2024 16:27 3875 activate.nu
-a---- 7/14/2024 16:27 2766 activate.ps1
-a---- 7/14/2024 16:27 2378 activate_this.py
-a---- 7/14/2024 16:27 1728 deactivate.bat
-a---- 7/13/2024 19:19 27648 libffi-8.dll
-a---- 7/13/2024 19:19 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 16:27 1215 pydoc.bat
-a---- 7/13/2024 19:19 79360 pypy.exe
-a---- 7/13/2024 19:19 79360 pypyw.exe
-a---- 7/13/2024 19:19 79360 python.exe
-a---- 7/13/2024 19:19 79360 pythonw.exe
```
## Test Plan
To verify the correct behaviour:
1. Download and install PyPy from [official
website](https://www.pypy.org/download.html)
2. Call `uv venv -p <path_to_pypy_>`
3. Run `.\.venv\Scripts\activate` on Windows or
`./.venv/Scripts/activate` on Unix
4. Run `pypy`
I thought of writing some automated tests but I couldn't rely on `uv
python install` command to install PyPy as it's not in the list of
installable Python builds.
In some cases, it's possible for the marker expressions on conflicting
dependency specification to be disjoint but *incomplete*. That is, if
one unions the disjoint markers, the result is not the complete set of
marker environments possible. There may be some "gap" of marker
environments not covered by the markers.
This is a problem in practice because, before this commit, we only
created forks in the resolver for specific marker expressions. So if a
dependency happened to fall in a "gap," our resolver would never see it.
This commit fixes this by adding a new split covering the negation of
the union of all marker expressions in a set of forks for a specific
package.
Originally, I had planned on only creating this split when it was known
that the gap actually existed. That is, when the negation of the marker
expressions did *not* correspond to the empty set. After a lot of
thought, unfortunately, this (I believe) effectively boils down to 3SAT,
which is NP-complete.
Instead, what we do here is *always* create an extra split unless we can
definitively tell that it is empty. We look for a few cases, but
otherwise throw our hands up and potentially do wasted work.
This also updates the lock scenario tests to reflect the actual bug fix
here.
An important update here is the new `fork-incomplete-markers` test.
The snapshot saved here is actually wrong; it is the bug we want to
fix. In particular, it is missing a dependency on `c` in its lock file
because it falls into a gap created by incomplete-but-disjoint marker
expressions.
The only pubgrub error that can occur is a `NoSolutionError`, and the
only place it can occur is `unit_propagation`, all other variants if
`PubGrubError` are unreachable. By changing the return type on pubgrub's
side (https://github.com/astral-sh/pubgrub/pull/28), we can remove the
pattern matching and the `unreachable!()` asserts on `PubGrubError`.
Our pubgrub error wrapper used to have a two phased initialization,
first mostly stubs in `solve[_tracked]()` and then adding the actual
context in `resolve()`. When constructing the error in `solve` we
already have all this context, so we can unify this to a regular
constructor and remove the special casing in `resolve()` and `hints()`.
Currently, with
```toml
[project]
name = "transformers"
version = "4.39.0.dev0"
requires-python = ">=3.10"
dependencies = [
"torch==1.10.0"
]
```
i get
```
$ uv sync --preview
Resolved 3 packages in 7ms
error: found distribution torch==1.10.0 @ registry+https://pypi.org/simple with neither wheels nor source distribution
```
This error message is wrong, there are wheels, they are just not
compatible. I initially got this error message during `uv lock` (in a
build), so i also added that this is about installation, not about
locking.
We should reject this version immediately because with the current
requires python, it can never be installed, but even then we need to
change the error message because you can be on the correct python
version, but an unsupported platform.
Fix#4988
## Summary
Running `uv python list` on glibc-based Linux will list musl pythons.
```bash
$ uv version
uv 0.2.24
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.11.9-linux-x86_64-musl <download available>
cpython-3.10.14-linux-x86_64-musl <download available>
cpython-3.9.19-linux-x86_64-musl <download available>
cpython-3.8.19-linux-x86_64-musl <download available>
cpython-3.7.9-linux-x86_64-musl <download available>
```
Change it to show Python matching the environment's libc as follows.
```bash
$ uv python list
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.11.9-linux-x86_64-gnu <download available>
cpython-3.10.14-linux-x86_64-gnu <download available>
cpython-3.9.19-linux-x86_64-gnu <download available>
cpython-3.8.19-linux-x86_64-gnu <download available>
cpython-3.7.9-linux-x86_64-gnu <download available>
```
Also, if --all-platforms is specified, change to list Python for all
architectures and libc.
```bash
$ uv python list --all-platforms
warning: `uv python list` is experimental and may change without warning.
cpython-3.12.3-windows-x86_64-none <download available>
cpython-3.12.3-windows-x86-none <download available>
cpython-3.12.3-macos-x86_64-none <download available>
cpython-3.12.3-macos-aarch64-none <download available>
cpython-3.12.3-linux-x86_64-musl <download available>
cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3
cpython-3.12.3-linux-x86_64-gnu /bin/python3
cpython-3.12.3-linux-x86_64-gnu <download available>
cpython-3.12.3-linux-s390x-gnu <download available>
cpython-3.12.3-linux-powerpc64le-gnu <download available>
cpython-3.12.3-linux-armv7-gnueabihf <download available>
cpython-3.12.3-linux-armv7-gnueabi <download available>
cpython-3.12.3-linux-aarch64-gnu <download available>
...
```
## Test Plan
The following commands were executed on the command line to confirm the
results in Ubuntu 24.04.
- `cargo run python list`
- `cargo run python list --all-platforms`
## Summary
Closes https://github.com/astral-sh/uv/issues/5040.
## Test Plan
```
❯ cargo run pip install torch torch-scatter --no-cache
⠼ torch-scatter==2.1.2 error: Failed to download and build `torch-scatter==2.1.2`
Caused by: Failed to build: `torch-scatter==2.1.2`
Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:
--- stderr:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
self.run_setup()
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/nt/6gf2v7_s3k13zq_t3944rwz40000gn/T/.tmpuxrhWj/builds-v0/.tmp1OBLbw/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'torch'
---
Caused by: This error likely indicates that torch-scatter==2.1.2 depends on torch, but doesn't declare it as a build dependency. If torch-scatter==2.1.2 is a first-party package, consider adding torch to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation
```
This excludes `--upgrade-package` from `compile_command` when value and
option are passed as a single argument. Eg:
```console
--upgrade-package=package
-P=package
-Ppackage
```
I missed this on #5032.
Fixes#5031.
## Test Plan
Tested locally
## Summary
Largely based on rustup's implementation (linked in the source).
Closes#5027.
## Test Plan
- Changed the executable directory to `uv/foo`.
- Ran script; verified that I could access executables in `foo`.
## Summary
Use the lockfile to prefill the `InMemoryIndex` used by the resolver.
This enables us to resolve completely from the lockfile without making
any network requests/builds if the requirements are unchanged. It also
means that if new requirements are added we can still avoid most I/O
during resolution, partially addressing
https://github.com/astral-sh/uv/issues/3925.
The main limitation of this PR is that resolution from the lockfile can
fail if new versions are requested that are not present in the lockfile,
in which case we have to perform a fresh resolution. Fixing this would
likely require lazy version/metadata requests by `VersionMap` (this is
different from the lazy parsing we do, the list of versions in a
`VersionMap` is currently immutable).
Resolves https://github.com/astral-sh/uv/issues/3892.
## Test Plan
Added a `deterministic!` macro that ensures that a resolve from the
lockfile and a clean resolve result in the same lockfile output for all
our current tests.
## Summary
Brings in some learnings from `rustup`:
fede22fea7/src/cli/self_update/shell.rs (L197).
For example: we only need to write to `.zshenv` (but we have to respect
`ZDOTDIR`). Additionally, for Fish, we need to respect `XDG_CONFIG_HOME`
## Summary
I'll open follow-up tickets for Windows support.
Closes https://github.com/astral-sh/uv/issues/4953.
## Test Plan
```
❯ cargo run tool install flask
Resolved 7 packages in 353ms
Prepared 7 packages in 392ms
Installed 7 packages in 17ms
+ blinker==1.8.2
+ click==8.1.7
+ flask==3.0.3
+ itsdangerous==2.2.0
+ jinja2==3.1.4
+ markupsafe==2.1.5
+ werkzeug==3.0.3
Installed 1 executable: flask
warning: /Users/crmarsh/.local/bin is not on your PATH. To use installed tools, run:
export PATH="/Users/crmarsh/.local/bin:$PATH"
```
Then:
```
❯ which flask
flask not found
```
Then:
```
❯ cargo run tool ensurepath
warning: `uv tool ensurepath` is experimental and may change without warning.
Updated configuration file: /Users/crmarsh/workspace/puffin/bar
Restart your shell for the changes to take effect.
```
Then:
```
❯ which flask
/Users/crmarsh/.local/bin/flask
```
## Summary
Resolves#5013.
## Test Plan
```console
❯ ./target/debug/uv tool run fastapi-cli
warning: `uv tool run` is experimental and may change without warning.
Resolved 9 packages in 28ms
The executable fastapi-cli was not found.
However, the following executables are available via uv tool run --from fastapi-cli <EXECUTABLE>:
- fastapi
```
```console
❯ ./target/debug/uvx fastapi-cli
warning: `uvx` is experimental and may change without warning.
Resolved 9 packages in 23ms
The executable fastapi-cli was not found.
However, the following executables are available via uvx --from fastapi-cli <EXECUTABLE>:
- fastapi
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Fixes#4941.
This PR adds a `--no-pager` option in `help` command to explicitly
disable the pager.
I noted that the template used for the text printed when calling `help`
with no argument or option doesn't show any option. It made sense before
this PR since `help` didn't have any available option. Though I'm unsure
if it makes sense to update the template as it would make it extremely
verbose as all the global options would be shown too.
I leave the decision to you.
## Test Plan
I ran `cargo run -- help` to verify `--isolated` was visible and it.
I ran clippy with `cargo clippy --workspace --all-targets --all-features
--locked -- -D warnings` as CI does.
I also ran tests locally with:
```
cargo nextest run \
--features python-patch \
--workspace \
--status-level skip --failure-output immediate-final --no-fail-fast -j 12 --final-status-level slow
```
[Doc tests can't use crate internal APIs unfortunately.][internal-doc]
I really want them to be able to, but for now, mark such tests as
`ignore`.
I was motivated to do this because it otherwise breaks `cargo t --all`
for me.
[internal-doc]: https://github.com/rust-lang/rust/issues/50784
## Summary
This PR makes the `--isolated` global argument visible, previously it
was hidden.
Fixes#4981.
## Test Plan
I ran `cargo run -- help` to verify `--isolated` was visible and it is.
I ran clippy with `cargo clippy --workspace --all-targets --all-features
--locked -- -D warnings` as CI does.
I also ran tests locally with:
```
cargo nextest run \
--features python-patch \
--workspace \
--status-level skip --failure-output immediate-final --no-fail-fast -j 12 --final-status-level slow
```
It's unclear to me whether this was intentional or not, but
I realized that converting a MarkerExpression to a string
treated EqualStar and NotEqualStar as Equal and NotEqual,
respectively. I tweaked this to match the Display impl for
VersionSpecifier.
(Negation tests in the next commit cover this change.)
## Summary
Converting to a lock requires that we generate hashes; but generating
hashes isn't required here. So let's just use a different representation
for the cache key.
Closes https://github.com/astral-sh/uv/issues/4990.
This is pulled out of #4632 — a user noted that it would be useful to
use the `uv` crate from Rust. This makes it way easier to invoke `uv`
from Rust with arbitrary arguments as well as use various functionality
in the `uv` crate.
Note this is no longer needed for #4632 and is not particularly urgent.
Adds a `uv python pin` command to write to a `.python-version` file.
We support all of our Python version request formats. We also support a
`--resolved` flag to pin to a specific interpreter instead of the
provided version. We canonicalize the request with #4949, it's not just
printed verbatim. We always attempt to find the interpreter so we can
warn if it's not available. With `--resolved`, if we can't find the
interpreter we fail. If no arguments are provided, we'll attempt to
display the current pin.
In the future:
- We should confirm that this satisfies the `Requires-Python` metadata
if a `pyproject.toml` is present
- We should support writing to a `uv.python-version` field if
`pyproject.toml` or `uv.toml` are present
- We should support finding and updating the "nearest" Python version
file (looking in ancestors)
- We should support finding version files in workspaces
- We should support some sort of global pin
## Summary
Closes https://github.com/astral-sh/uv/issues/4671.
## Test Plan
```
❯ XDG_BIN_HOME="/Users/crmarsh/workspace/uv/foo bar" cargo run tool install black --force
Installed 2 executables: black, blackd
warning: `/Users/crmarsh/workspace/uv/foo bar` is not on your PATH. To use installed tools, run:
export PATH="/Users/crmarsh/workspace/uv/foo bar:$PATH"
```
Needed over in https://github.com/astral-sh/uv/pull/4674
These filters are relatively aggressive and may have a high false
positive rate, we don't need them for most tests so we leave them as
opt-in for now.
Special cases the `Any` request in output
e.g.,
```
❯ cargo run -q -- python install --isolated
warning: `uv python install` is experimental and may change without warning.
Searching for Python installations
Found existing installation: cpython-3.12.3-macos-aarch64-none
Python is already available. Use `uv python install <request>` to install a specific version.
```
instead of
```
❯ cargo run -q -- python install --isolated
warning: `uv python install` is experimental and may change without warning.
Searching for Python versions matching: any Python
Found existing installation for any Python: cpython-3.12.3-macos-aarch64-none
Python is already available. Use `uv python install <request>` to install a specific version.
```
## Summary
We currently store wheel URLs in an unparsed state because we don't have
a stable parsed representation to use with rykv. Unfortunately this
means we end up reparsing unnecessarily in a lot of places, especially
when constructing a `Lock`. This PR adds a `UrlString` type that lets us
avoid reparsing without losing the validity of the `Url`.
## Test Plan
Shaves off another ~10 ms from
https://github.com/astral-sh/uv/issues/4860.
```
➜ transformers hyperfine "../../uv/target/profiling/uv lock" "../../uv/target/profiling/baseline lock" --warmup 3
Benchmark 1: ../../uv/target/profiling/uv lock
Time (mean ± σ): 120.9 ms ± 2.5 ms [User: 126.0 ms, System: 80.6 ms]
Range (min … max): 116.8 ms … 125.7 ms 23 runs
Benchmark 2: ../../uv/target/profiling/baseline lock
Time (mean ± σ): 129.9 ms ± 4.2 ms [User: 127.1 ms, System: 86.1 ms]
Range (min … max): 123.4 ms … 141.2 ms 23 runs
Summary
../../uv/target/profiling/uv lock ran
1.07 ± 0.04 times faster than ../../uv/target/profiling/baseline lock
```
## Summary
Move completely off tokio's multi-threaded runtime. We've slowly been
making changes to be smarter about scheduling in various places instead
of depending on tokio's general purpose work-stealing, notably
https://github.com/astral-sh/uv/pull/3627 and
https://github.com/astral-sh/uv/pull/4004. We now no longer benefit from
the multi-threaded runtime, as we run on all I/O on the main thread.
There's one remaining instance of `block_in_place` that can be swapped
for `rayon::spawn`.
This change is a small performance improvement due to removing some
unnecessary overhead of the multi-threaded runtime (e.g. spawning
threads), but nothing major. It also removes some noise from profiles.
## Test Plan
```
Benchmark 1: ./target/profiling/uv (resolve-warm)
Time (mean ± σ): 14.9 ms ± 0.3 ms [User: 3.0 ms, System: 17.3 ms]
Range (min … max): 14.1 ms … 15.8 ms 169 runs
Benchmark 2: ./target/profiling/baseline (resolve-warm)
Time (mean ± σ): 16.1 ms ± 0.3 ms [User: 3.9 ms, System: 18.7 ms]
Range (min … max): 15.1 ms … 17.3 ms 162 runs
Summary
./target/profiling/uv (resolve-warm) ran
1.08 ± 0.03 times faster than ./target/profiling/baseline (resolve-warm)
```
## Summary
Avoid serializing and writing the lockfile if a cheap comparison shows
that the contents have not changed.
## Test Plan
Shaves ~10ms off of https://github.com/astral-sh/uv/issues/4860 for me.
```
➜ transformers hyperfine "../../uv/target/profiling/uv lock" "../../uv/target/profiling/baseline lock" --warmup 3
Benchmark 1: ../../uv/target/profiling/uv lock
Time (mean ± σ): 130.5 ms ± 2.5 ms [User: 130.3 ms, System: 85.0 ms]
Range (min … max): 126.8 ms … 136.9 ms 23 runs
Benchmark 2: ../../uv/target/profiling/baseline lock
Time (mean ± σ): 140.5 ms ± 5.0 ms [User: 142.8 ms, System: 85.5 ms]
Range (min … max): 133.2 ms … 153.3 ms 21 runs
Summary
../../uv/target/profiling/uv lock ran
1.08 ± 0.04 times faster than ../../uv/target/profiling/baseline lock
```
Adds a nice hint at the bottom of `uv help` output indicating how to get
more details about a specific command, roughly matching Cargo's
interface.
We use the short help and skip the pager for the root `uv help` since
it's intended to be a landing page for the help interface more than an
in-depth display. This also matches Cargo, though I like that they have
the global options above the commands and I've not changed that here.
## Summary
We now recreate the environment in `uv sync`, `uv tool install`, and `uv
tool run` if the underlying interpreter has been removed.
Closes https://github.com/astral-sh/uv/issues/4933.
This is an attempt to solve https://github.com/astral-sh/uv/issues/ by
applying the extra marker of the requirement to overrides and
constraints.
Say in `a` we have a requirements
```
b==1; python_version < "3.10"
c==1; extra == "feature"
```
and overrides
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10"
c==3; python_version >= "3.10"
```
Our current strategy is to discard the markers in the original
requirements. This means that on 3.12 for `a` we install `b==3`, but it
also means that we add `c` to `a` without `a[feature]`, causing #4826.
With this PR, the new requirement become,
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10" and extra == "feature"
c==3; python_version >= "3.10" and extra == "feature"
```
allowing to override markers while preserving optional dependencies as
such.
Fixes#4826
## Summary
Allows `--all` as an alternative to specifying specific targets.
## Test Plan
Verified that `cargo run python uninstall` still fails.
```
❯ cargo run python uninstall --all
Compiling uv-cli v0.0.1 (/Users/crmarsh/workspace/puffin/crates/uv-cli)
Compiling uv v0.2.23 (/Users/crmarsh/workspace/puffin/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.86s
Running `target/debug/uv python uninstall --all`
warning: `uv python uninstall` is experimental and may change without warning.
Searching for Python installations
Found existing installation: cpython-3.9.18-macos-aarch64-none
Found existing installation: cpython-3.8.18-macos-aarch64-none
Found existing installation: cpython-3.8.12-macos-aarch64-none
Found existing installation: cpython-3.12.1-macos-aarch64-none
Found existing installation: cpython-3.11.7-macos-aarch64-none
Found existing installation: cpython-3.10.13-macos-aarch64-none
Uninstalled cpython-3.10.13-macos-aarch64-none
Uninstalled cpython-3.11.7-macos-aarch64-none
Uninstalled cpython-3.12.1-macos-aarch64-none
Uninstalled cpython-3.8.12-macos-aarch64-none
Uninstalled cpython-3.8.18-macos-aarch64-none
Uninstalled cpython-3.9.18-macos-aarch64-none
Uninstalled 6 versions in 479ms
```
I feel like I'm always drowning in the help output from `uv` because we
have so many options.
I basically agree with the commentary in
https://github.com/clap-rs/clap/issues/4687 that having different
behaviors for `-h` and `--help` is surprising. I think `--help` is more
obvious for users and I want to optimize for that experience.
This roughly matches the help menus in Cargo and pip.
The `uv help` command can be used for long help. In #4906 and #4909 we
improve that command.
Extends #4904 which adds test cases for the existing behavior.
## Summary
In marker normalization, we now remove any markers that are redundant
with the `requires-python` specifier (i.e., always true for the given
Python requirement).
For example, given `iniconfig ; python_version >= '3.7'`, we can remove
the `python_version >= '3.7'` marker when resolving with
`--python-version 3.8`.
Closes#4852.
## Summary
Given `python_version != '3.8' and python_version < '3.10'`, the first
term was expanded to `python_version < '3.8'` and `python_version >
'3.8'`. We then AND'd all three terms together. We don't seem to have a
way to differentiate between the terms to AND and the terms to OR in the
normalization code (it all gets flattened together), so instead this PR
expands the expressions at the leaf level and then flattens them at the
level above when appropriate.
Closes https://github.com/astral-sh/uv/issues/4910.
## Summary
Closes https://github.com/astral-sh/uv/issues/4915.
```
error: No interpreter found for Python 3.12.4 in virtual environments, managed installations, or system path
```
## Summary
Closes https://github.com/astral-sh/uv/issues/4848.
## Test Plan
```
> cargo run -- run -vv --preview --isolated --python 3.12.4 python -V
error: No interpreter found for Python 3.12.4 in virtual environments or managed installations or system path
```
## Summary
More marker simplification:
- Filters out redundant subtrees based on outer expressions, e.g. `a and (a or
b)` simplifies to `a`.
- Flattens nested trees internally, e.g. `(a and b) and c`
Resolves https://github.com/astral-sh/uv/issues/4536.
The PR #4707 introduced the notion of "version narrowing," where a
Requires-Python constraint was _possibly_ narrowed whenever the
universal resolver created a fork. The version narrowing would occur
when the fork was a result of a marker expression on `python_version`
that is *stricter* than the configured `Requires-Python` (via, say,
`pyproject.toml`).
The crucial conceptual change made by #4707 is therefore that
`Requires-Python` is no longer an invariant configuration of resolution,
but rather a mutable constraint that can vary from fork to fork. This in
turn can result in some cases, such as in #4885, where different
versions of dependencies are selected. We aren't sure whether we can fix
those or not, with version narrowing, so for now, we do this revert to
restore the previous behavior and we'll try to address the version
narrowing some other time.
This also adds the case from #4885 as a regression test, ensuring that
we don't break that in the future. I confirmed that with version
narrowing, this test outputs duplicate distributions. Without narrowing,
there are no duplicates.
Ref #4707, Fixes#4885
This test was, I believe, relying on the XDG_DATA_HOME environment
variable not being set. When it is set, as is the case in my
environment, `uv tool run` will respect it and install `black` for this
particular test into my actual XDG_DATA_HOME directory. We fix this by
setting `XDG_DATA_HOME` explicitly.
By using `Box::pin(run())` we can reduce the artificial stack size for
running tests on windows in debug mode from 8MB to 2MB. I've checked and
1MB/no custom stack size still fail tests, e.g.
`add_workspace_editable`.
This test is failing most times for me when running nextest locally,
failing the overall test run, so i'm deactivating it for now. I'm still
not sure what the root cause here is. It seems to have something to do
with python stdin not being ready immediately after we spawn the process
and us being too fast.
## Summary
Resolves#4834
## Test Plan
```sh
# 3.12.3 is a `install_only` archive
$ cargo run -- python install --preview --force 3.12.3
# 3.9.4 has only `full` archive
$ cargo run -- python install --preview --force 3.9.4
```
## Summary
Partially closes#1917
This PR picks up on some of the great work from #1864 and opted to keep
`panic_immediate_abort` (for size reasons). I split the PR in different
isolated commits in case we want to separate/cherry-pick them out.
1. The first commit ports mostly all std changes from that PR into this
PR. Binary sizes stayed the same ~16kb.
2. The second commit migrates our existing usage of windows-sys to
windows for a safer ffi calls with Results!. It also changes all large
unsafe blocks to be isolated to the actual unsafe calls, and switches
some areas to use std such as getenv port ( which seemed buggy! ) from
launcher.c. In addition, this also adds more error checking in order to
match some missing assertions from distlib's launcher.c. Note, due to
the additional .text data, the binary sizes increased to ~20.5kb, but we
can cut back on some of the added error msgs as needed.
3. The third commit switches to using xwin for building on all 3
supported trampoline targets for sanity, and adds a CI bloat check for
core::fmt and panic as a precaution. Sadly, this will invalidate the
xwin cache on the first run.
## Test Plan
Most changes were tested on a couple of local GUI apps and console apps,
also tested some of the error states manually by using SetLastError at
different points in the code and/or passing in invalid handles.
I'm not sure how far we can get with migrating some of the other calls
without increasing binary size substantially. An initial attempt at
using std::path didn't seem so bad size wise when I tried it (~1k). On
other cases, such as std::process::exit added ~10k to the total binary
size.
---------
Co-authored-by: konstin <konstin@mailbox.org>
## Summary
If you pass `--isolated` but no `--with`, at present, we don't create
any environment (so `--python` isn't respected and `python` will fail
entirely if it wasn't already in your path). Now, we create a base
environment in `--isolated` even if `with` wasn't provided.
Closes https://github.com/astral-sh/uv/issues/4846.
Closes https://github.com/astral-sh/uv/issues/4776.
## Summary
Like https://github.com/astral-sh/uv/pull/4808 but with a few more
changes. I suspect this will require some bikeshedding but I find the
use of "installation" and "installed" in the same sentence to be kind of
a lot.
## Summary
There are a few ideas at play here:
1. pip always strips versions to the release when evaluating against a
`Requires-Python`, so we now do the same. That means, e.g., using
`3.13.0b0` will be accepted by a project with `Requires-Python: >=
3.13`, which does _not_ adhere to PEP 440 semantics but is somewhat
intuitive.
2. Because we know we'll only be evaluating against release-only
versions, we can use different semantics in PubGrub that let us collapse
ranges. For example, `python_version >= '3.10' or python_version <
'3.10'` can be collapsed to the truthy marker.
Closes https://github.com/astral-sh/uv/issues/4714.
Closes https://github.com/astral-sh/uv/issues/4272.
Closes https://github.com/astral-sh/uv/issues/4719.
Remove wheels from the lockfile that don't match the required python
version. For example, we remove
`charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl` when we have
`requires-python = ">=3.12"`.
Our snapshots barely show changes since we avoid the large binaries for
which matters. Here are 3 real world `uv.lock` before/after comparisons
to show the large difference:
*
[warehouse](https://gist.github.com/konstin/9a1ed6a32b410e250fcf4c6ea8c536a5)
(5677 -> 4214)
*
[transformers](https://gist.github.com/konstin/5636281b5226f64aa44ce3244d5230cd)
(6484 -> 5816)
*
[github-wikidata-bot](https://gist.github.com/konstin/ebbd7b9474523aaa61d9a8945bc02071)
(793 -> 454)
We only remove wheels we are certain don't match the python version and
still keep those with unknown tags. We could remove even more wheels by
also considering other markers, e.g. removing linux wheels for a
windows-only dep, but we would trade complex, easy-to-get-wrong logic
for diminishing returns.
## Summary
Check the sha256 checksum when downloading a managed python toolchain.
## Test Plan
```sh
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12.3 (any-3.12.3-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
Installed Python 3.12.3 to C:\Users\jo\AppData\Roaming\uv\data\python\cpython-3.12.3-windows-x86_64-none
Installed 1 installation in 6s
$ cargo run -- python uninstall 3.12
$ # manually change the hash in `crates/uv-python/src/downloads.inc`
$ cargo run -- python install 3.12
warning: `uv python install` is experimental and may change without warning.
Looking for installation Python 3.12 (any-3.12-any-any-any)
Downloading cpython-3.12.3-windows-x86_64-none
error: Hash mismatch for `cpython-3.12.3-windows-x86_64-none`
Expected:
xx
Computed:
776568c92c5f3b47dbf5f17c1c58578f70d75a32654419a158aa8bdc6f95b09a
```
## Summary
This seems like another good candidate for environment caching. If you
run a script repeatedly, we can just use the existing cached
environment.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Addresses https://github.com/astral-sh/uv/issues/4330, to reduce
duplication in the client creation logic.
## Test Plan
<!-- How was it tested? -->
https://github.com/astral-sh/uv/pull/4729#issuecomment-2204681655
## Summary
The resolver sometimes starts HTTP requests that end up not being
necessary. When dropping the Tokio runtime before exiting we currently
wait for those to complete. This can cause noticeable hangs in the CLI,
particularly when the runtime is blocked on slow DNS resolution.
Resolves https://github.com/astral-sh/uv/issues/4599.
## Test Plan
This change resolves any reproducible hangs for me locally.
## Summary
The basic strategy:
- When the user does `uv tool run`, we resolve the `from` and `with`
requirements (always).
- After resolving, we generate a hash of the requirements. For now, I'm
just converting to a lockfile and hashing _that_, but that's an
implementation detail.
- Once we have a hash, we _also_ hash the interpreter.
- We then store environments in
`${CACHE_DIR}/${INTERPRETER_HASH}/${RESOLUTION_HASH}`.
Some consequences:
- We cache based on the interpreter, so if you request a different
Python, we'll create a new environment (even if they're compatible).
This has the nice side-effect of ensuring that we don't use environments
for interpreters that were later deleted.
- We cache the `from` and `with` together. In practice, we may want to
cache them separately, then layer them? But this is also an
implementation detail that we could change later.
- Because we use the lockfile as the cache key, we will invalidate the
cache when the format changes. That seems ok, but we could improve it in
the future by generating a stable hash from a lockfile that's
independent of the schema.
Closes https://github.com/astral-sh/uv/issues/4752.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes#4653
## Summary
Adds the tool version to the list command right beside the tool name
```
$ uv tool list
black v24.2.0
```
Following the proposed format discussed in #4653
## Test Plan
`cargo test tool_list`
<!-- How was it tested? -->
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
The changes in https://github.com/astral-sh/uv/pull/4708 caused an
overflow in debug mode only of the 1MB default stack size in windows
during clap. This means that even trivial wrong argument tests would
fail without increasing the stack size. As remedy, we box the clap
types.
This fell out of my investigation of
https://github.com/astral-sh/uv/issues/4774 but the bug was fixed by the
reporter in #4775
- Adds support for `GH_TOKEN` authentication again — basically needed to
avoid rate limits when hacking on this.
- Clarifies some handling and logging of flavors
## Summary
This doesn't cache the tool environment; rather, it just uses the `tool
install` environment if it satisfies the request.
Closes https://github.com/astral-sh/uv/issues/4742.
Previously this displayed:
```
❯ cargo run -q -- --help
The command line interface for the uv binary.
Usage: uv [OPTIONS] <COMMAND>
```
This is.. weird. Here I remove it entirely. I could see adding
`about_long` text being helpful in the future.
Fix#4774.
## Summary
Change the python interpreter for linux installed with `uv python` to an
optimized one.
## Test Plan
I ran the following command on Linux (glibc) to confirm that an
optimized (not debug built) Python is installed.
```bash
# install python
uv python install 3.12.3
# check build type
uv run python -c "import sysconfig;print(sysconfig.get_config_var('Py_DEBUG'))"
0
```
## Summary
This used to be necessary because we purged the cache in the
`InstallPlan` if the user passed `--reinstall`. _However_, we later
changed the cache to be append-only.
## Test Plan
I ran through the test plan in https://github.com/astral-sh/uv/pull/933,
which includes an integration test and running `uv pip install
--reinstall` with:
```text
setuptools
devpi @ e334eb4dc9bb023329e4b610e4515b/devpi-2.2.0.tar.gz
```
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes#4654
## Summary
The purpose of this is to show the entrypoints of each tool when running
`uv tool list` as below:
```
$ uv tool list
black
black
blackd
```
I used the proposed formatting as it was written in #4653 by @blueraft.
I had to use spaces instead of tabs in order to make the test
successful. Indeed in the test we are using a raw string and I did not
manage to make the test pass when escaping the tab in the list.rs file
so I used spaces everywhere.
I had a deeper look into #4653 as well but it is more difficult as we
need to get the version of the tool in the Tool object, I will continue
on this next one later.
Please tell me if anything else is needed I tried to follow the
contribution guidelines but I might have forgotten something.
Have a great day!
## Test Plan
`cargo clippy`
then by using the local version of uv as described in the Readme.md.
```
my-computer :~/mypath/uv$ cargo run -- tool list
Compiling uv-cli v0.0.1 (/mypath/uv/crates/uv-cli)
Compiling uv v0.2.18 (/mypath/uv/crates/uv)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.69s
Running `target/debug/uv tool list`
warning: `uv tool list` is experimental and may change without warning.
black
black
blackd
isort
isort
isort-identify-imports
```
and
`cargo test tool_list`
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
For now the semantics are such that if the requested requirements from
the command line don't match the receipt (or if any `--reinstall` or
`--upgrade` is requested), we proceed with an install, passing the
`--reinstall` and `--upgrade` to the underlying Python environment.
This may lead to some unintuitive behaviors, but it's simplest for now.
For example:
- `uv tool install black<24` followed by `uv tool install black
--upgrade` will install the latest version of `black`, removing the
`<24` constraint.
- `uv tool install black --with black-plugin` followed by `uv tool
install black` will remove `black-plugin`.
Closes https://github.com/astral-sh/uv/issues/4659.
In #3514 and #2755, users had intermittent network errors, but it was
not always clear whether we had already retried these requests or not.
Building upon https://github.com/TrueLayer/reqwest-middleware/pull/159,
this PR adds the number of retries to the error message, so we can see
at first glance where we're missing retries and where we might need to
change retry settings.
Example error trace:
```
Could not connect, are you offline?
Caused by: Request failed after 3 retries
Caused by: error sending request for url (https://pypi.org/simple/uv/)
Caused by: client error (Connect)
Caused by: dns error: failed to lookup address information: Name or service not known
Caused by: failed to lookup address information: Name or service not known
```
This code is ugly since i'm missing a better pattern for attaching
context to reqwest middleware errors in
https://github.com/TrueLayer/reqwest-middleware/pull/159.
## Summary
Given:
```text
numpy >=1.26 ; python_version >= '3.9'
numpy <1.26 ; python_version < '3.9'
```
When resolving for Python 3.8, we need to narrow the `requires-python`
requirement in the top branch of the fork, because `numpy >=1.26` all
require Python 3.9 or later -- but we know (in that branch) that we only
need to _solve_ for Python 3.9 or later.
Closes https://github.com/astral-sh/uv/issues/4669.
## Summary
This is required to solve https://github.com/astral-sh/uv/issues/4669,
because the `Requires-Python` version can now vary across a resolution.
For example, within certain forks, we might have a more narrow range,
which would allow us to use distributions that would not be allowed for
the global resolution.
This should be fine because `requires-python` is part of the package
metadata, so it should be consistent between files within a package
version. As such, there shouldn't be any risk that we incorrectly
prioritize distributions by omitting this information.
(To be more specific, the risk is something like: we prioritize some
wheel over a source distribution within a package-version, so we don't
track the source distribution at all. Then, later, when we choose a
candidate, we see that the wheel doesn't meet the `Requires-Python`
requirement, even though the source distribution _would've_ met it. If
files within a distribution could have varied support, this would be a
real risk.)
I think `--toolchain-preference system` is sufficiently clear and
`--toolchain-preference prefer-system` is excessively verbose. This was
discussed in the original pull request at
https://github.com/astral-sh/uv/pull/4424 but because we had a case for
preferring "installed managed" toolchains I was hesitant to change it.
Now that I've dropped that in #4601, I think we can drop the prefix.
Adds a `toolchain-fetch` option alongside `toolchain-preference` with
`automatic` (default) and `manual` values allowing automatic toolchain
fetches to be disabled (replaces
https://github.com/astral-sh/uv/pull/4425). When `manual`, toolchains
must be installed with `uv toolchain install`.
Note this was previously implemented with `if-necessary`, `always`,
`never` variants but the interaction between this and
`toolchain-preference` was too confusing. By reducing to a binary
option, things should be clearer. The `if-necessary` behavior moved to
`toolchain-preference=installed`. See
https://github.com/astral-sh/uv/pull/4601#discussion_r1657839633 and
https://github.com/astral-sh/uv/pull/4601#discussion_r1658658755
Closes https://github.com/astral-sh/uv/issues/4476
Originally, this used the changes in #4642 to invoke `main()` from a
`uvx` binary. This had the benefit of `uvx` being entirely standalone at
the cost of doubling our artifact size. We think that's the incorrect
trade-off.
Instead, we assume `uvx` is always next to `uv` and create a tiny binary
(<1MB) that invokes `uv` in a child process. This seems preferable to a
`cargo-dist` alias because we have more control over it. This binary
should "just work" for all of our cargo-dist distributions and
installers, but we'll need to add a new entry point for our PyPI
distribution. I'll probably tackle support there separately?
```
❯ ls -lah target/release/uv target/release/uvx
-rwxr-xr-x 1 zb staff 31M Jun 28 23:23 target/release/uv
-rwxr-xr-x 1 zb staff 452K Jun 28 23:22 target/release/uvx
```
This includes some small overhead:
```
❯ hyperfine --shell=none --warmup=100 './target/release/uv tool run --help' './target/release/uvx --help' --min-runs 2000
Benchmark 1: ./target/release/uv tool run --help
Time (mean ± σ): 2.2 ms ± 0.1 ms [User: 1.3 ms, System: 0.5 ms]
Range (min … max): 2.0 ms … 4.0 ms 2000 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: ./target/release/uvx --help
Time (mean ± σ): 2.9 ms ± 0.1 ms [User: 1.7 ms, System: 0.9 ms]
Range (min … max): 2.8 ms … 4.2 ms 2000 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
./target/release/uv tool run --help ran
1.35 ± 0.09 times faster than ./target/release/uvx --help
```
I presume there may be some other downsides to a child process? The
wrapper is a little awkward. We could consider `execv` but this is
complicated across platforms. An example implementation of that over in
[monotrail](433af5aed9/crates/monotrail/src/monotrail.rs (L764-L799)).
## Summary
I ended up needing this for https://github.com/astral-sh/uv/issues/4664
but I think it's a good change more broadly. We should be able to share
this cached information across operations within a given invocation.
## Summary
These are changing in one of my branches but I can't tell _what's_
changing. Some tests include the lock, but others don't. This PR adds it
for all successful resolves in the suite.
## Summary
You can now add `managed = false` under `[tool.uv]` in a
`pyproject.toml` to explicitly opt out of the project and workspace
APIs.
If a project sets `managed = false`, we will (1) _not_ discover it as a
workspace root, and (2) _not_ discover it as a workspace member (similar
to using `exclude` in the workspace parent).
Closes https://github.com/astral-sh/uv/issues/4551.
## Summary
This doesn't actually change any behaviors, but it does make it a bit
easier to solve #4669, because we don't have to support "version
narrowing" for the non-`RequiresPython` variants in here. Right now, the
semantics are kind of muddied, because the `target` variant is
_sometimes_ interpreted as an exact version and sometimes as a lower
bound.
## Summary
`GitDatabase::contains` previously only parsed the commit to see if it
was a valid hash and didn't verify if the commit existed in the object
database. This led to the database never being updated.
Resolves https://github.com/astral-sh/uv/issues/4378.
## Test Plan
Added a test that fails without this change.
It's hard to talk about solve state and resolver state, so i'm renaming
them to fork state and resolver state, indicating the hierarchy between
more directly.
## Summary
Closes https://github.com/astral-sh/uv/issues/4688.
## Test Plan
```
❯ cargo run tool install ruff
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/uv tool install ruff`
warning: `uv tool install` is experimental and may change without warning.
Resolved 1 package in 136ms
Installed 1 package in 3ms
+ ruff==0.5.0
No entrypoints to install for tool `ruff`
```
## Summary
Packages that provide scripts that _aren't_ Python entrypoints need to
respected in `uv tool install`. For example, Ruff ships a script in
`ruff-0.5.0.data/scripts`.
Unfortunately, the `.data` directory doesn't exist in the virtual
environment at all (it's removed, per the spec, after install). So this
PR changes the entry point detection to look at the `RECORD` file, which
is the only evidence that the scripts were installed.
Closes https://github.com/astral-sh/uv/issues/4691.
## Test Plan
`cargo run uv tool install ruff` (snapshot tests to-come)
## Summary
Resolves#4483Resolves#4484
## Test Plan
`cargo test`
```sh
❯ cargo run -- toolchain dir
warning: `uv toolchain dir` is experimental and may change without warning.
/Users/ahmedilyas/Library/Application Support/uv/toolchains
❯ cargo run -- tool dir
warning: `uv tool dir` is experimental and may change without warning.
/Users/ahmedilyas/Library/Application Support/uv/tools
```
## Summary
I think this may have just been a typo.
Closes https://github.com/astral-sh/uv/issues/4692.
## Test Plan
Run `cargo run tool install flask --force --reinstall` repeatedly.
## Summary
I noticed that `init_environment` and `find_interpreter` were both
calling `find_environment`, which seemed like a code smell to me.
Instead, `find_interpreter` now returns either a compatible environment
or an interpreter (if no compatible environment was found).
Additionally, `interpreter_meets_requirements` now no longer validates
`requires-python` if `--python` or `.python-version` is set. Instead, we
warn, which matches the behavior we get when creating a new environment
at the bottom of `find_interpreter`.
In total, I think this makes the data flow in project interpreter
discovery less repetitive and easier to reason about.
## Summary
This should both make it faster to solve forks (since we have a guess
for a valid resolution, and will bias towards packages we've already
fetched) and improve consistency between forks.
Closes https://github.com/astral-sh/uv/issues/4617.
Charlie Marsh
Ibraheem Ahmed
Zanie Blue
konsti
Chan Kang
Ahmed Ilyas
samypr100
Delgan
renovate[bot]
FishAlchemist
eth3lbert
Pavel Dikov
Andrew Gallant
Vigilans
Ben Beasley
Maksim Bondarenkov
bluss
Krishnan Chandra
Jo
Trevor Manz
Zack Elia
Yorick
Di-Is
Mathieu Kniewallner
Danny
Tim Felgentreff
Silvano Cerza
Ivan Shcheklein
skshetry
messense
Alexander Gherm
Caíque Porfirio
Billy Doyle
pppp56
David Poznik