Python/uv

mirror of https://github.com/astral-sh/uv synced 2026-01-23 22:40:10 -05:00

Files

Zanie Blue 44e39bdca3 Replace Python bootstrapping script with Rust implementation (#2842 )

See https://github.com/astral-sh/uv/issues/2617

Note this also includes:
- #2918 
- #2931 (pending)

A first step towards Python toolchain management in Rust.

First, we add a new crate to manage Python download metadata:

- Adds a new `uv-toolchain` crate
- Adds Rust structs for Python version download metadata
- Duplicates the script which downloads Python version metadata
- Adds a script to generate Rust code from the JSON metadata
- Adds a utility to download and extract the Python version

I explored some alternatives like a build script using things like
`serde` and `uneval` to automatically construct the code from our
structs but deemed it to heavy. Unlike Rye, I don't generate the Rust
directly from the web requests and have an intermediate JSON layer to
speed up iteration on the Rust types.

Next, we add add a `uv-dev` command `fetch-python` to download Python
versions per the bootstrapping script.

- Downloads a requested version or reads from `.python-versions`
- Extracts to `UV_BOOTSTRAP_DIR`
- Links executables for path extension

This command is not really intended to be user facing, but it's a good
PoC for the `uv-toolchain` API. Hash checking (via the sha256) isn't
implemented yet, we can do that in a follow-up.

Finally, we remove the `scripts/bootstrap` directory, update CI to use
the new command, and update the CONTRIBUTING docs.

<img width="1023" alt="Screenshot 2024-04-08 at 17 12 15"
src="https://github.com/astral-sh/uv/assets/2586601/57bd3cf1-7477-4bb8-a8e9-802a00d772cb">

2024-04-10 11:22:41 -05:00

4.5 KiB

Raw Blame History

Contributing

We have issues labeled as Good First Issue and Help Wanted which are good opportunities for new contributors.

Setup

Rust, a C compiler, and CMake are required to build uv.

Linux

On Ubuntu and other Debian-based distributions, you can install the C compiler and CMake with

sudo apt install build-essential cmake

macOS

CMake may be installed with Homebrew:

brew install cmake

See the Python section for instructions on installing the Python versions.

Windows

You can install CMake from the installers or with pipx install cmake (make sure that the pipx install path is in PATH, pipx complains if it isn't).

Testing

For running tests, we recommend nextest.

Python

Testing uv requires multiple specific Python versions. You can install them into <project root>/bin via our bootstrapping script:

cargo run -p uv-dev -- fetch-python

You may need to add the versions to your PATH:

source .env

You can configure the bootstrapping directory with UV_BOOTSTRAP_DIR.

Local testing

You can invoke your development version of uv with cargo run -- <args>. For example:

cargo run -- venv
cargo run -- pip install requests

Running inside a docker container

Source distributions can run arbitrary code on build and can make unwanted modifications to your system ("Someone's Been Messing With My Subnormals!" on Blogspot, "nvidia-pyindex" on PyPI), which can even occur when just resolving requirements. To prevent this, there's a Docker container you can run commands in:

docker buildx build -t uv-builder -f builder.dockerfile --load .
# Build for musl to avoid glibc errors, might not be required with your OS version
cargo build --target x86_64-unknown-linux-musl --profile profiling --features vendored-openssl
docker run --rm -it -v $(pwd):/app uv-builder /app/target/x86_64-unknown-linux-musl/profiling/uv-dev resolve-many --cache-dir /app/cache-docker /app/scripts/popular_packages/pypi_10k_most_dependents.txt

We recommend using this container if you don't trust the dependency tree of the package(s) you are trying to resolve or install.

Profiling and Benchmarking

Please refer to Ruff's Profiling Guide, it applies to uv, too.

We provide diverse sets of requirements for testing and benchmarking the resolver in scripts/requirements and for the installer in scripts/requirements/compiled.

You can use scripts/bench to benchmark predefined workloads between uv versions and with other tools, e.g.

python -m scripts.bench \
    --uv-path ./target/release/before \
    --uv-path ./target/release/after \
    ./scripts/requirements/jupyter.in --benchmark resolve-cold --min-runs 20

Analysing concurrency

You can use tracing-durations-export to visualize parallel requests and find any spots where uv is CPU-bound. Example usage, with uv and uv-dev respectively:

RUST_LOG=uv=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --profile profiling -- pip compile scripts/requirements/jupyter.in

RUST_LOG=uv=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --bin uv-dev --profile profiling -- resolve jupyter

Trace-level logging

You can enable trace level logging using the RUST_LOG environment variable, i.e.

RUST_LOG=trace uv …

Releases

Releases can only be performed by Astral team members.

Changelog entries and version bumps are automated. First, run:

./scripts/release.sh

Then, editorialize the CHANGELOG.md file to ensure entries are consistently styled.

Then, open a pull request e.g. Bump version to ....

Binary builds will automatically be tested for the release.

After merging the pull request, run the release workflow with the version tag. Do not include a leading v. The release will automatically be created on GitHub after everything else publishes.

4.5 KiB Raw Blame History