1c7c174bc8
This fixes an obscure cache collision in Python interpreter queries, which we believe to be the root cause of CI flakes we've been seeing where a project environment is invalidated and recreated. This work follows from the logs in [this CI run](https://github.com/astral-sh/uv/actions/runs/15934322410/job/44950599993?pr=14326) which captured one of the flakes with tracing enabled. There, we can see that the project environment is invalidated because the Python interpreter in the environment has a different version than expected: ``` DEBUG Checking for Python environment at `.venv` TRACE Cached interpreter info for Python 3.12.9, skipping probing: .venv/bin/python3 DEBUG The interpreter in the project environment has different version (3.12.9) than it was created with (3.9.21) ``` (this message is updated to reflect #14329) The flow is roughly: - We create an environment with 3.12.9 - We query the environment, and cache the interpreter version for `.venv/bin/python` - We create an environment for 3.9.12, replacing the existing one - We query the environment, and read the cached information The Python cache entries are keyed by the absolute path to the interpreter, and rely on the modification time (ctime, nsec resolution) of the canonicalized path to determine if the cache entry should be invalidated. The key is a hex representation of a u64 sea hasher output — which is very unlikely to collide. After an audit of the Python query caching logic, we determined that the most likely cause of a collision in cache entries is that the modification times of underlying interpreters are identical. This seems pretty feasible, especially if the file system does not support nanosecond precision — though it appears that the GitHub runners do support it. The fix here is to include the canonicalized path in the cache key, which ensures we're looking at the modification time of the _same_ underlying interpreter. This will "invalidate" all existing interpreter cache entries but that's not a big deal. This should also have the effect of reducing cache churn for interpreters in virtual environments. Now, when you change Python versions, we won't invalidate the previous cache entry so if you change _back_ to the old version we can re-use our cached information. It's a bit speculative, since we don't have a deterministic reproduction in CI, but this is the strongest candidate given the logs and should increase correctness regardless. Closes https://github.com/astral-sh/uv/issues/14160 Closes https://github.com/astral-sh/uv/issues/13744 Closes https://github.com/astral-sh/uv/issues/13745 Once it's confirmed the flakes are resolved, we should revert - https://github.com/astral-sh/uv/pull/14275 - #13817 |
||
|---|---|---|
| .. | ||
| uv | ||
| uv-auth | ||
| uv-bench | ||
| uv-build | ||
| uv-build-backend | ||
| uv-build-frontend | ||
| uv-cache | ||
| uv-cache-info | ||
| uv-cache-key | ||
| uv-cli | ||
| uv-client | ||
| uv-configuration | ||
| uv-console | ||
| uv-dev | ||
| uv-dirs | ||
| uv-dispatch | ||
| uv-distribution | ||
| uv-distribution-filename | ||
| uv-distribution-types | ||
| uv-extract | ||
| uv-fs | ||
| uv-git | ||
| uv-git-types | ||
| uv-globfilter | ||
| uv-install-wheel | ||
| uv-installer | ||
| uv-macros | ||
| uv-metadata | ||
| uv-normalize | ||
| uv-once-map | ||
| uv-options-metadata | ||
| uv-pep440 | ||
| uv-pep508 | ||
| uv-performance-memory-allocator | ||
| uv-platform-tags | ||
| uv-publish | ||
| uv-pypi-types | ||
| uv-python | ||
| uv-redacted | ||
| uv-requirements | ||
| uv-requirements-txt | ||
| uv-resolver | ||
| uv-scripts | ||
| uv-settings | ||
| uv-shell | ||
| uv-small-str | ||
| uv-state | ||
| uv-static | ||
| uv-tool | ||
| uv-torch | ||
| uv-trampoline | ||
| uv-trampoline-builder | ||
| uv-types | ||
| uv-version | ||
| uv-virtualenv | ||
| uv-warnings | ||
| uv-workspace | ||
| README.md | ||
README.md
Crates
uv-bench
Functionality for benchmarking uv.
uv-cache-key
Generic functionality for caching paths, URLs, and other resources across platforms.
uv-distribution-filename
Parse built distribution (wheel) and source distribution (sdist) filenames to extract structured metadata.
uv-distribution-types
Abstractions for representing built distributions (wheels) and source distributions (sdists), and the sources from which they can be downloaded.
uv-install-wheel-rs
Install built distributions (wheels) into a virtual environment.
uv-once-map
A waitmap-like concurrent hash map for executing tasks
exactly once.
uv-pep440-rs
Utilities for interacting with Python version numbers and specifiers.
uv-pep508-rs
Utilities for parsing and evaluating dependency specifiers, previously known as PEP 508.
uv-platform-tags
Functionality for parsing and inferring Python platform tags as per PEP 425.
uv-cli
Command-line interface for the uv package manager.
uv-build-frontend
A PEP 517-compatible build frontend for uv.
uv-cache
Functionality for caching Python packages and associated metadata.
uv-client
Client for interacting with PyPI-compatible HTTP APIs.
uv-dev
Development utilities for uv.
uv-dispatch
A centralized struct for resolving and building source distributions in isolated environments.
Implements the traits defined in uv-types.
uv-distribution
Client for interacting with built distributions (wheels) and source distributions (sdists). Capable of fetching metadata, distribution contents, etc.
uv-extract
Utilities for extracting files from archives.
uv-fs
Utilities for interacting with the filesystem.
uv-git
Functionality for interacting with Git repositories.
uv-installer
Functionality for installing Python packages into a virtual environment.
uv-python
Functionality for detecting and leveraging the current Python interpreter.
uv-normalize
Normalize package and extra names as per Python specifications.
uv-requirements
Utilities for reading package requirements from pyproject.toml and requirements.txt files.
uv-resolver
Functionality for resolving Python packages and their dependencies.
uv-shell
Utilities for detecting and manipulating shell environments.
uv-types
Shared traits for uv, to avoid circular dependencies.
uv-pypi-types
General-purpose type definitions for types used in PyPI-compatible APIs.
uv-virtualenv
A venv replacement to create virtual environments in Rust.
uv-warnings
User-facing warnings for uv.
uv-workspace
Workspace abstractions for uv.
uv-requirements-txt
Functionality for parsing requirements.txt files.