When a process is running and another calls `uv cache clean` or `uv
cache prune` we currently deadlock - sometimes until the CI timeout
(https://github.com/astral-sh/setup-uv/issues/588). To avoid this, we
add a default 5 min timeout waiting for a lock. 5 min balances allowing
in-progress builds to finish, especially with larger native
dependencies, while also giving timely errors for deadlocks on (remote)
systems.
Commit 1 is a refactoring.
This branch also fixes a problem with the logging where acquired and
released resources currently mismatch:
```
DEBUG Acquired lock for `https://github.com/tqdm/tqdm`
DEBUG Using existing Git source `https://github.com/tqdm/tqdm`
DEBUG Released lock at `C:\Users\Konsti\AppData\Local\uv\cache\git-v0\locks\16bb813afef8edd2`
```
This removes executable permissions while retaining global read / global
write.
It's been suggested we should use 0o644 instead, dropping the global
write permissions (i.e., just the owner can write), but since we're
taking an exclusive lock I don't think that would work and we'd regress
the issue that was solved by updating the permissions. I think we'll
need to revisit the locking scheme if that's the goal, but regardless,
this seems like a net improvement.
Part of https://github.com/astral-sh/uv/issues/4392
We shouldn't link to PyPI, and dropping the workspace-level
documentation link should mean that we get the auto-generated `docs.rs`
links.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR removes the crate fs2 and updates Rust version to 1.89.
*Why?*
Crate fs2 is unmaintained for a long time now and has unfixed issues.
Especially it doesn't build on AIX, which is the reason I started fixing
it.
*How?*
I removed fs2 and replaced it by std:fs:File methods.
## Test Plan
<!-- How was it tested? -->
- I built it on Windows and AIX only.
- I did not test the artifacts.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Currently, `uv cache clean` and `uv cache prune` can cause crashes in
other uv processes running in parallel by removing their in-use files.
We can solve this by using a shared (read) lock on the cache directory,
while the `uv cache` operations use an exclusive (write) lock. The
drawback is that this is always one extra lock, and that we assume that
all platforms support shared locks.
Once Rust 1.89 fulfills our N-2 policy, we can add support for these
methods in fs_err and switch to
https://doc.rust-lang.org/std/fs/struct.File.html#platform-specific-behavior-2.
**Test Plan**
Open one terminal, run:
```
uv venv -c -p 3.13
UV_CACHE_DIR=cache uv cache clean
UV_CACHE_DIR=cache uv pip install numpy==2.0.0
```
Open another terminal, run:
```
UV_CACHE_DIR=cache uv cache clean
```
Fixes#15704
Part of #13883
The initial motivation for this change was that we were using both the
`windows`, the `window_sys` and the `windows_core` crate in various
places. These crates have slightly unconventional versioning scheme
where there is a large workspace with the same version in general, but
only some crates get breaking releases when a new breaking release
happens, the others stay on the previous breaking version. The `windows`
crate is a shim for all three of them, with a single version. This
simplifies handling the versions.
Using `windows` over `windows_sys` has the advantage of a higher level
error interface, we now get a `Result` for all windows API calls instead
of C-style int-returns and get-last-error calls. This makes the
uv-keyring crate more resilient.
We keep using the `windows_registry` crate, which provides a higher
level interface to windows registry access.
Adds locking of the credentials store for concurrency safety. It's
important to hold the lock from read -> write so credentials are not
dropped during concurrent writes.
I opted not to attach the lock to the store itself. Instead, I return
the lock on read and require it on write to encourage safe use. Maybe
attaching the source path to the store struct and adding a `lock(&self)`
method would make sense? but then you can forget to take the lock at the
right time. The main problem with the interface here is to write a _new_
store you have to take the lock yourself, and you could make a mistake
by taking a lock for the wrong path or something. The fix for that would
be to introduce a new `CredentialStoreHandle` type or something, but
that seems overzealous rn. We also don't eagerly drop the lock on token
read, although we could.
This is an alternative to https://github.com/astral-sh/uv/pull/14788
which has the benefit that it addresses
https://github.com/astral-sh/uv/issues/13327 which would be an issue
even if we reverted #14447.
There are two changes here
1. We copy entry points into the ephemeral environment, and rewrite
their shebangs (or trampoline target) to ensure the ephemeral
environment is not bypassed.
2. We link `etc/jupyter` and `share/jupyter` data directories into the
ephemeral environment, this is in order to ensure the above doesn't
break Jupyter which unfortunately cannot find the `share` directory
otherwise. I'd love not to do this, as it seems brittle and we don't
have a motivating use-case beyond Jupyter. I've opened
https://github.com/jupyterlab/jupyterlab/issues/17716 upstream for
discussion, as there is a viable patch that could be made upstream to
resolve the problem. I've limited the fix to Jupyter directories so we
can remove it without breakage.
Closes https://github.com/astral-sh/uv/issues/14729
Closes https://github.com/astral-sh/uv/issues/13327
Closes https://github.com/astral-sh/uv/issues/14749
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
This is a continuation of the work in
* #12405
I have:
* moved to an architecture where the human output is derived from the
json structs to centralize more of the printing state/logic
* cleaned up some of the names/types
* added tests
* removed the restriction that this output is --dry-run only
I have not yet added package info, which was TBD in their design.
---------
Co-authored-by: x0rw <mahdi.svt5@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: John Mumm <jtfmumm@gmail.com>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Update [schemars
0.9.0](https://github.com/GREsau/schemars/releases/tag/v0.9.0)
There are differences in the generated JSON Schema and I will [contact
the author](https://github.com/GREsau/schemars/issues/407).
## Test Plan
---------
Co-authored-by: konstin <konstin@mailbox.org>
We were checking whether a path was an executable in a virtual
environment or the base directory of a virtual environment in multiple
places in the codebase. This PR consolidates this logic into one place.
Closes#13947.
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
We've been using a number of different winapi crates. This PR removes
winsafe in favor of the official windows-* crates, so all of uv's own
winapi calls go through the official windows-* crates.
---------
Co-authored-by: Aria Desires <aria.desires@gmail.com>
In #13302, there was an IO error without context. This error seems to be
caused by a symlink error. Switching as symlinking to `fs_err` ensures
these errors will carry context in the future.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
This PR moves functions for finding user- and system-level config
directories to public functions in `uv_fs::config`. This will allow them
to be used in future work without duplicating code.
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.
In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
Instead of using junctions, we can just write files that contain (as the
file contents) the target path. This requires a little more finesse in
that, as readers, we need to know where to expect these. But it also
means we get to avoid junctions, which have led to a variety of
confusing behaviors. Further, `replace_symlink` should now be on atomic
on Windows.
Closes#11263.
## Summary
These are used for coordination across processes. If you run uv under,
e.g., the root user, then under a different user, I don't think we
should prevent you from acquiring the lock.
Closes https://github.com/astral-sh/uv/issues/11324.
## Summary
This lets us drop a dependency entirely. `percent-encoding` is used by
`url` and so is already in the graph, whereas `urlencoding` isn't used
by anything else.
## Summary
On Windows, we have a lot of issues with atomic replacement and such.
There are a bunch of different failure modes, but they generally
involve: trying to persist a fail to a path at which the file already
exists, trying to replace or remove a file while someone else is reading
it, etc.
This PR adds locks to all of the relevant database paths. We already use
these advisory locks when building source distributions; now we use them
when unzipping wheels, storing metadata, etc.
Closes#11002.
## Test Plan
I ran the following script:
```shell
# Define the cache directory path
$cacheDir = "C:\Users\crmar\workspace\uv\cache"
# Clear the cache directory if it exists
if (Test-Path $cacheDir) {
Remove-Item -Recurse -Force $cacheDir
}
# Create the cache directory again
New-Item -ItemType Directory -Force -Path $cacheDir
# Define the command to run with --cache-dir flag
$command = {
param ($venvPath)
# Create a virtual environment in the specified path with --python
uv venv $venvPath
# Run the pip install command with --cache-dir flag
C:\Users\crmar\workspace\uv\target\profiling\uv.exe pip install flask==1.0.4 --no-binary flask --cache-dir C:\Users\crmar\workspace\uv\cache -v --python $venvPath
}
# Define the paths for the different virtual environments
$venv1 = "C:\Users\crmar\workspace\uv\venv1"
$venv2 = "C:\Users\crmar\workspace\uv\venv2"
$venv3 = "C:\Users\crmar\workspace\uv\venv3"
$venv4 = "C:\Users\crmar\workspace\uv\venv4"
$venv5 = "C:\Users\crmar\workspace\uv\venv5"
# Start the command in parallel five times using Start-Job, each with a different venv
$job1 = Start-Job -ScriptBlock $command -ArgumentList $venv1
$job2 = Start-Job -ScriptBlock $command -ArgumentList $venv2
$job3 = Start-Job -ScriptBlock $command -ArgumentList $venv3
$job4 = Start-Job -ScriptBlock $command -ArgumentList $venv4
$job5 = Start-Job -ScriptBlock $command -ArgumentList $venv5
# Wait for all jobs to complete
$jobs = @($job1, $job2, $job3, $job4, $job5)
$jobs | ForEach-Object { Wait-Job $_ }
# Retrieve the results (optional)
$jobs | ForEach-Object { Receive-Job -Job $_ }
# Clean up the jobs
$jobs | ForEach-Object { Remove-Job -Job $_ }
```
And ensured it succeeded in five straight invocations (whereas on
`main`, it consistently fails with a variety of different traces).
This should be essentially the exact same behaviour, but backon is a
total API redesign, so things had to be expressed slightly differently.
Overall I think the code is more readable, which is nice.
Fixes#10001
## Summary
A few places where there are extra conversions to and from string that
seem unnecessary; a few places where we're using `PathBuf` instead of
`PortablePathBuf`.
Zanie Blue
konsti
Zsolt Dollenstein
Michael Richter
samypr100
Aria Desires
Charlie Marsh
Jack O'Connor
ya7010
John Mumm
Jo