Files
uv/crates
konsti fb4ba2bbc2 Speed up cold cache urllib3/boto3/botocore with batched prefetching (#2452)
With pubgrub being fast for complex ranges, we can now compute the next
n candidates without taking a performance hit. This speeds up cold cache
`urllib3<1.25.4` `boto3` from maybe 40s - 50s to ~2s. See docstrings for
details on the heuristics.

**Before**


![image](https://github.com/astral-sh/uv/assets/6826232/b7b06950-e45b-4c49-b65e-ae19fe9888cc)

**After**


![image](https://github.com/astral-sh/uv/assets/6826232/1c749248-850e-49c1-9d57-a7d78f87b3aa)

---

We need two parts of the prefetching, first looking for compatible
version and then falling back to flat next versions. After we selected a
boto3 version, there is only one compatible botocore version remaining,
so when won't find other compatible candidates for prefetching. We see
this as a pattern where we only prefetch boto3 (stack bars), but not
botocore (sequential requests between the stacked bars).


![image](https://github.com/astral-sh/uv/assets/6826232/e5186800-23ac-4ed1-99b9-4d1046fbd03a)

The risk is that we're completely wrong with the guess and cause a lot
of useless network requests. I think this is acceptable since this
mechanism only triggers when we're already on the bad path and we should
simply have fetched all versions after some seconds (assuming a fast
index like pypi).

---

It would be even better if the pubgrub state was copy-on-write so we
could simulate more progress than we actually have; currently we're
guessing what the next version is which could be completely wrong, but i
think this is still a valuable heuristic.

Fixes #170.
2024-04-08 14:28:56 +00:00
..
2024-02-11 18:53:58 +00:00
2024-04-07 23:42:19 -05:00
2024-02-15 11:19:46 -06:00

Crates

bench

Functionality for benchmarking uv.

cache-key

Generic functionality for caching paths, URLs, and other resources across platforms.

distribution-filename

Parse built distribution (wheel) and source distribution (sdist) filenames to extract structured metadata.

distribution-types

Abstractions for representing built distributions (wheels) and source distributions (sdists), and the sources from which they can be downloaded.

install-wheel-rs

Install built distributions (wheels) into a virtual environment.]

once-map

A waitmap-like concurrent hash map for executing tasks exactly once.

pep440-rs

Utilities for interacting with Python version numbers and specifiers.

pep508-rs

Utilities for interacting with PEP 508 dependency specifiers.

platform-host

Functionality for detecting the current platform (operating system, architecture, etc.).

platform-tags

Functionality for parsing and inferring Python platform tags as per PEP 425.

uv

Command-line interface for the uv package manager.

uv-build

A PEP 517-compatible build frontend for uv.

uv-cache

Functionality for caching Python packages and associated metadata.

uv-client

Client for interacting with PyPI-compatible HTTP APIs.

uv-dev

Development utilities for uv.

uv-dispatch

A centralized struct for resolving and building source distributions in isolated environments. Implements the traits defined in uv-types.

uv-distribution

Client for interacting with built distributions (wheels) and source distributions (sdists). Capable of fetching metadata, distribution contents, etc.

uv-extract

Utilities for extracting files from archives.

uv-fs

Utilities for interacting with the filesystem.

uv-git

Functionality for interacting with Git repositories.

uv-installer

Functionality for installing Python packages into a virtual environment.

uv-interpreter

Functionality for detecting and leveraging the current Python interpreter.

uv-normalize

Normalize package and extra names as per Python specifications.

uv-package

Types and functionality for working with Python packages, e.g., parsing wheel files.

uv-requirements

Utilities for reading package requirements from pyproject.toml and requirements.txt files.

uv-resolver

Functionality for resolving Python packages and their dependencies.

uv-types

Shared traits for uv, to avoid circular dependencies.

pypi-types

General-purpose type definitions for types used in PyPI-compatible APIs.

uv-virtualenv

A venv replacement to create virtual environments in Rust.

uv-warnings

User-facing warnings for uv.

requirements-txt

Functionality for parsing requirements.txt files.