Also took the time to migrate to the external config format to normalize
our projects for team comfort (`ty` *has* to use this format for its
workspace structure).
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
This PR is a combination of #12920 and #13754. Prior to these changes,
following a redirect when searching indexes would bypass our
authentication middleware. This PR updates uv to support propagating
credentials through our middleware on same-origin redirects and to
support netrc credentials for both same- and cross-origin redirects. It
does not handle the case described in #11097 where the redirect location
itself includes credentials (e.g.,
`https://user:pass@redirect-location.com`). That will be addressed in
follow-up work.
This includes unit tests for the new redirect logic and integration
tests for credential propagation. The automated external registries test
is also passing for AWS CodeArtifact, Azure Artifacts, GCP Artifact
Registry, JFrog Artifactory, GitLab, Cloudsmith, and Gemfury.
Replaces https://github.com/astral-sh/uv/pull/12320
Switches to Depot for the large Windows runner we use for `cargo test`.
The runtime goes from 8m 20s -> 6m 44s (total) and 7m 18s -> 4m 41s
(test run) which are 20% and 35% speedups respectively.
A few things got marginally slower, like Python installs went from 11s
-> 38s, the Rust cache went from 15s -> 30s, and drive setup went from
7s -> 20s.
Close#13922
## Summary
Add a warning if the directory given by the `--index` argument is empty.
## Test Plan
Added test case `add_index_empty_directory` in `edit.rs`
The script stumbled over a newline introduced in
https://github.com/pypi/warehouse/pull/18266 (which is valid).
Also fixed: Don't read versions for the same package from other indexes.
We were using `project_name` here instead of `target`, while using the
latter and only reading from a single index simplifies the code too.
This PR provides a script that uses environment variables to determine
which registries to test. This script is being used to run automated
registry tests in CI for AWS, Azure, GCP, Artifactory, GitLab,
Cloudsmith, and Gemfury.
You must configure the following required env vars for each registry:
```
UV_TEST_<registry_name>_URL URL for the registry
UV_TEST_<registry_name>_TOKEN authentication token
UV_TEST_<registry_name>_PKG private package to install
```
The username defaults to "\_\_token\_\_" but can be optionally set with:
```
UV_TEST_<registry_name>_USERNAME
```
For each configured registry, the test will attempt to install the
specified package. Some registries can fall back to PyPI internally, so
it's important to choose a package that only exists in the registry you
are testing.
Currently, a successful test means that it finds the line “ +
<package_name>” in the output. This is because in its current form we
don’t know ahead of time what package it is and hence what the exact
expected output would be. The advantage if that anyone can run this
locally, though they would have to have access to the registries they
want to test.
You can also use the `--use-op` command line argument to derive these
test env vars from a 1Password vault (default is "RegistryTests" but can
be configured with `--op-vault`). It will look at all items in the vault
with names following the pattern `UV_TEST_<registry_name>` and will
derive the env vars as follows:
```
`UV_TEST_<registry_name>_USERNAME` from the `username` field
`UV_TEST_<registry_name>_TOKEN` from the `password` field
`UV_TEST_<registry_name>_URL` from a field with the label `url`
`UV_TEST_<registry_name>_PKG` from a field with the label `pkg`
```
When working on support for reading global Python pins in tool
operations, I noticed that we weren't using the canonicalized Python
request in receipts — we were using the raw string provided by the user.
Since we'll need to compare these values, we should be using the
canonicalized string.
The `Tool` and `ToolReceipt` types have been updated to hold a
`PythonRequest` instead of a `String`, and `Serialize` was implemented
for `PythonRequest` so canonicalization can happen at the edge instead
of being the caller's responsibility.
The dist plan parsing is pretty hard to understand, and I want to add
more images, e.g., for DockerHub in #14088. As a simplifying
precursor... move the dist plan processing into a dedicated step.
Fix `uv run -p 3.7` by not using a walrus operator. Python 3.7 isn't
really supported anymore, but there's no reason to break interpreter
discovery for it.
When using `uv lock --upgrade-package=python` after changing
`requires-python`, it was possible to get into a state where the fork
markers produced corresponded to the empty set. This in turn resulted in
an empty lock file.
There was already some infrastructure in place that I think was perhaps
intended to handle this. In particular, `Lock::check_marker_coverage`
checks whether the fork markers have some overlap with the supported
environments (including the `requires-python`). But there were two
problems with this.
First is that in lock validation, this marker coverage check came
_after_ a path that returned `Preferable` (meaning that the fork markers
should be kept) when `--upgrade-package` was used. Second is that the
marker coverage check used the `requires-python` in the lock file and
_not_ the `requires-python` in the now updated `pyproject.toml`.
We attempt to solve this conundrum by slightly re-arranging lock file
validation and by explicitly checking whether the *new*
`requires-python` is disjoint from the fork markers in the lock file. If
it is, then we return `Versions` from lock file validation (indicating
that the fork markers should be dropped).
Fixes#13951
We always ignore the `clippy::struct_excessive_bools` rule and formerly
annotated this at the function level. This PR specifies the allow in
`workspace.lints.clippy` in `Cargo.toml`.
## Summary
Simplify the Docker image build process to leverage Depot container
builders for faster layer caching and native multi-platform image
builds. The combo of the two removes the need to save cache to
registries and do complex merge operations across GHA runners to get a
multi-platform image.
## Test Plan
UV team will need to add a trust relationship in Depot so that the
container builds can authenticate and run. This can be done following
these docs:
https://depot.dev/docs/cli/authentication#adding-a-trust-relationship-for-github-actions.
Once that is done, this should just work as before, but without all of
the extra work around manifests. We should double that all of the
tagging still makes sense for you all, as some bits of that were
unclear.
Additional context in this draft PR:
https://github.com/astral-sh/uv/pull/9156
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
I only just realized that we can get backtraces for crashes with
`RUST_BACKTRACE: 1`, even non-panics
(https://github.com/astral-sh/uv/pull/14079). This is much better than
trying to analyze crash dumps.
We've gotten away without this file for a while. In particular, we
explicitly use its default settings.
However, this is occasionally problematic in certain contexts where
`rustfmt` is invoked directly. Or in contexts where the Rust Edition is
otherwise not specified. At least, this happens when using the Rust vim
plugin. When an edition isn't explicitly specified, it defaults back to
the 2015 edition.
I think that there aren't a lot of rustfmt changes, and so we've been
able to get away with this for a while. But it looks like something in
the 2024 edition changes how imports are ordered. So to make it explicit
that we want to use the 2024 edition of rustfmt, we opt into it.
This is analogous to a change made to the Ruff repository somewhat
recently: https://github.com/astral-sh/ruff/pull/18197
Using a companion change in the middleware
(https://github.com/TrueLayer/reqwest-middleware/pull/235, forked&tagged
pending review), we can check and show retries for HTTP status core
errors, to consistently report retries again.
We fix two cases:
* Show retries for status code errors for cache client requests
* Show retries for status code errors for Python download requests
Not handled:
* Show previous retries when a distribution download fails mid-streaming
* Perform retries when a distribution download fails mid-streaming
* Show previous retries when a Python download fails mid-streaming
* Perform retries when a Python download fails mid-streaming
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[acj/freebsd-firecracker-action](https://redirect.github.com/acj/freebsd-firecracker-action)
| action | minor | `v0.4.2` -> `v0.5.0` |
---
> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.
---
### Release Notes
<details>
<summary>acj/freebsd-firecracker-action
(acj/freebsd-firecracker-action)</summary>
###
[`v0.5.0`](https://redirect.github.com/acj/freebsd-firecracker-action/releases/tag/v0.5.0)
[Compare
Source](https://redirect.github.com/acj/freebsd-firecracker-action/compare/v0.4.2...v0.5.0)
Changes:
- Add `disk-size` option to control the size of the VM's disk and root
filesystem
- Retired obsolete workaround that disabled TCP segmentation offload
(TSO)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/uv).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC41MC4wIiwidXBkYXRlZEluVmVyIjoiNDAuNTAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [which](https://redirect.github.com/harryfei/which-rs) |
workspace.dependencies | major | `7.0.0` -> `8.0.0` |
---
> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.
---
### Release Notes
<details>
<summary>harryfei/which-rs (which)</summary>
###
[`v8.0.0`](https://redirect.github.com/harryfei/which-rs/blob/HEAD/CHANGELOG.md#800)
[Compare
Source](https://redirect.github.com/harryfei/which-rs/compare/7.0.3...8.0.0)
- Add new `Sys` trait to allow abstracting over the underlying
filesystem. Particularly useful for `wasm32-unknown-unknown` targets.
Thanks [@​dsherret](https://redirect.github.com/dsherret) for this
contribution to which!
- Add more debug level tracing for otherwise silent I/O errors.
- Call the `NonFatalHandler` in more places to catch previously ignored
I/O errors.
- Remove use of the `either` dependency.
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/uv).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC41MC4wIiwidXBkYXRlZEluVmVyIjoiNDAuNTAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
(or legacy tool.uv.workspace).
This cleaves out a dedicated SourcedDependencyGroups type based on
RequiresDist but with only the DependencyGroup handling implemented.
This allows `uv pip` to read `dependency-groups` from pyproject.tomls
that only have that table defined, per PEP 735, and as implemented by
`pip`.
However we want our implementation to respect various uv features when
they're available:
* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)
As such we want to opportunistically detect "as much as possible" while
doing as little as possible when things are missing. The issue with the
old RequiresDist path was that it fundamentally wanted to build the
package, and if `[project]` was missing it would try to desperately run
setuptools on the pyproject.toml to try to find metadata and make a hash
of things.
At the same time, the old code also put in a lot of effort to try to
pretend that `uv pip` dependency-groups worked like `uv`
dependency-groups with defaults and non-only semantics, only to separate
them back out again. By explicitly separating them out, we confidently
get the expected behaviour.
Note that dependency-group support is still included in RequiresDist, as
some `uv` paths still use it. It's unclear to me if those paths want
this same treatment -- for now I conclude no.
Fixes#13138
This allows you to specify requires-python on individual dependency-groups,
with the intended usecase being "oh my dev-dependencies have a higher
requires-python than my actual project".
This includes a large driveby move of the RequiresPython type to
uv-distribution-types to allow us to generate the appropriate markers at
this point in the code. It also migrates RequiresPython from
pubgrub::Range to version_ranges::Ranges, and makes several pub(crate)
items pub, as it's no longer defined in uv_resolver.
Fixes#11606
This brings in https://github.com/BurntSushi/jiff/pull/385, which makes
cold resolves about 10% faster. Or, stated differently, as fast as they
were a few weeks ago before the perf regression introduced by
`jiff 0.2.14`.
Aria Desires
John Mumm
Jack O'Connor
Jack O'Connor
Zanie Blue
konsti
Charlie Marsh
Aaron Ang
Andrew Gallant
FishAlchemist
Kyle Galbraith
renovate[bot]
Frazer McLean