## Summary
This is blocking the release (#9793). We seem to have hit some sort of
limit that's causing builds to fail on this target. It's a Tier 3 Rust
target with _unknown_ (???) `std` support (see the question mark
[here](https://doc.rust-lang.org/rustc/platform-support.html)).
When trying to upload without a password but with the keyring, check
that the keyring has a password for the upload URL and username and warn
if it doesn't.
Fixes#8781
## Summary
In the Windows Clippy job, the workspace is transferred to
`UV_WORKSPACE`. However, `cargo clippy` continues to execute in the
`github.workspace`, and `Swatinem/rust-cache` only caches the
`UV_WORKSPACE/target`, resulting in `cargo clippy` having no cache.
This adjustment will take effect when any changes are made to
`Cargo.toml` or `Cargo.lock`, prompting `Swatinem/rust-cache` to updat
the cache.
Incorporating #8637 into #8458
- Adds `python-managed` feature selection to Windows CI for `python
install` tests
- Adds trampoline sniffing utilities to `uv-trampoline-builder`
- Uses a trampoline to install Python executables into the `PATH` on
Windows
Updates `uv python install` to link `python3.x` in the executable
directory (i.e., `~/.local/bin`) to the the managed interpreter path.
Includes
- #8569
- #8571
Remaining work
- #8663
- #8650
- Add an opt-out setting and flag
- Update documentation
Currently, our trampoline is used to convert `<command> [args]` to
`python <command> [args]` for script entrypoints installed into virtual
environments. For #8458, it'd be nice to convert a shim `python3.12
[args]` to `python [args]`. Here, we modify the trampolines to support
this use-case.
The only change we really need here is to avoid injecting `<command>`
into the child process. We change the "magic number" at the end of the
trampoline executables from `UVUV` to `UVSC` and `UVPY` which define
"script" and "python" variants to the trampoline. We then omit the
`<command>` injection in the latter case. We also omit writing the zip
script payload.
To support construction of the new variant, a new
`uv-trampoline-builder` crate is introduced — this avoids requirements
on `uv-install-wheel` in future work. I also use `uv-trampoline-builder`
to consolidate some of the test setup for `uv-trampoline`.
There should be no backwards compatibility concerns, since trampolines
are fully self-referential.
I rebased to fix the commits at the end, as this took many iterations to
get working via CI. This should roughly be reviewable by commit if you
prefer.
It seems unintentional that we basically did nothing on these
alternative platforms? It seems like an artifact from some previous
change.
I'm not sure it's worth running Clippy multiple times. We could also
just reduce the matrix here.
Cherry-picked from #8347
Might fix https://github.com/astral-sh/uv/issues/6940 — I'm not seeing a
failure over there after this change. I think there may be some problem
with concurrent reads of junctioned files on the DevDrive? It's really
hard to say.
We might lose some important test coverage with this change. I'm not
sure what to do about that either.
Closes https://github.com/astral-sh/uv/issues/8228
e.g., on this branch
```
❯ uv python install 3.13t 3.13
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13 --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13t --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
and on main
```
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
Installed 3 packages in 12ms
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
I want to add more test coverage around this, but I've noticed the
free-threaded discovery tests are a bit off as-is and it'll be a bigger
task. I think the recent bugs around discovery indicate we should invest
more into that test framework.
## Summary
Reverts #8181 and #8182.
The fix is in b849f0f, which extends the run timeout to allow xwin to
download the Windows SDK files, which can take 10+ minutes.
Closes https://github.com/rust-cross/cargo-xwin/issues/127
## Test Plan
Existing CI should pass.
## Notes
xwin jobs will take a long time the first time due to cache re-warming.
## Summary
Update CI config to test against GraalPy 24.1, which makes a workaround
in the CI config unnecessary.
## Test Plan
The updated CI config is tested.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
I started learning `uv` by inspecting the source code.
I've noticed that your shell scripts are very good! Which is rare!
## Test Plan
I propose to add `shellcheck` to the CI.
It is a great tool to help finding bugs and style issues in shell code.
Techincal details:
- This CI job will only run when any `.sh` files are changed (or the job
definition file)
- It takes just several seconds even on local machine:
```
» time shellcheck -S style **/*.sh
shellcheck -S style **/*.sh 0.02s user 0.05s system 61% cpu 0.123 total
```
- It is easy to use, for example: I just fixed the single problem you
had in your code with `# shellcheck disable=SC1091`
- I am using this tool for around 8 years now and didn't have any
issues. Examples:
ca899f3b69/.github/workflows/test.yml (L22-L27)
and
https://github.com/wemake-services/wemake-django-template/blob/master/.github/workflows/shellcheck.yml
But, I understand that build / lint tools are very subjective. So, feel
free to close :)
## Summary
Closes https://github.com/astral-sh/uv/issues/6972
This is not breaking since `MACOSX_DEPLOYMENT_TARGET` will stay the same
(currently defaulting to 10.12) so a
`uv-x.y.z-py3-none-macosx_10_12_x86_64.whl` will still be built
## Summary
Copy both `uv` and `uvx` into place in a single Dockerfile command.
[`COPY` supports multiple sources when the destination is a
directory.](https://docs.docker.com/engine/reference/builder/#copy)
As it is, e.g. `ghcr.io/astral-sh/uv:0.4.16-python3.12-bookworm-slim`
has this (screenshot from [Dive](https://github.com/wagoodman/dive)):
<img width="377" alt="Screenshot 2024-09-26 at 10 11 24"
src="https://github.com/user-attachments/assets/1ca6a0d5-95fd-4210-9a4f-0afa2300b63f">
and less layers is a Good Thing.
## Test Plan
I hope the CI pipeline will take care of testing – I couldn't get the
Docker build to finish on my machine right away (SIGKILL, so out of
memory, I guess 😄)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
closes#4828
First iteration for an implementation. I need to add more tests but
wanted your opinion on the implementation first.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Currently tested using the following command but will add tests shortly:
```console
D:\repo\uv> cargo run venv -p 3.13t && .venv\Scripts\python.exe
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
Running `target\debug\uv.exe venv -p 3.13t`
Using Python 3.13.0rc1 interpreter at: C:\Users\bschoen\AppData\Local\Programs\Python\Python313\python3.13t.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e78, Jul 31 2024, 21:06:58) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Originally wanted to update the reference to `astral-sh/setup-uv` in
https://docs.astral.sh/uv/guides/integration/github/, but thought it
could be nice to automate those updates through Renovate. The custom
manager will look for any major version GitHub Action reference in any
Markdown file in `docs` directory, and raise a PR to update it.
Possible improvements:
- We could separate those updates from updating the actions updates for
uv's own GitHub Actions workflow, which would end up raising 2 different
PRs instead of grouping them (example of the current behaviour without
this improvement in
https://github.com/mkniewallner/mkv-playground/pull/4 where we update
the doc reference at the same time as a real dependency usage in a
workflow).
- ~Should the PRs be raised immediately, to handle the update as soon as
possible, instead of waiting for the regular weekly Monday schedule?
This would ensure that `astral-sh/setup-uv` references are handled as
early as possible.~ done in
6af7f45750
## Test Plan
I've tested that with
00ddfb6900/renovate.json5
and
00ddfb6900/docs/integeration/foo.md,
where Renovate raised 2 PRs:
- https://github.com/mkniewallner/mkv-playground/pull/13
- https://github.com/mkniewallner/mkv-playground/pull/4
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
Closes https://github.com/astral-sh/uv/issues/7458
This change adds a new job at the end of docker publish which adds
annotations the primary image digests in order to make ghcr.io rank
`ghcr.io/astral-sh/uv` at the top once again. The solution is to
annotate the index at the end during a re-publish to make ghcr.io
consider it a more updated entry than the others and rank it at the top
once again.
## Test Plan
Tested on release run on my own fork
* Packages: https://github.com/samypr100/uv/pkgs/container/uv will show
`ghcr.io/astral-sh/uv` first once again
* Run: https://github.com/samypr100/uv/actions/runs/10951404736
## Summary
Because a problem was found with Powershell and combining the generated
completion scripts for uv and uvx, let's try separating uv and uvx
command completion scripts.
The generated powershell script template can be seen in clap_complete
source, and it starts with `using` directives, which makes it impossible
(apparently) to concatenate two of those script outputs.
As a side effect, this is available under `uv tool run
--generate-shell-completion` too.
Fixes#7482
## Test Plan
- `eval "$(cargo run --bin uvx -- --generate-shell-completion bash)"`
- Test Powershell
Similar to our semantics for packages with pre-release versions.
We will not use prerelease versions unless there are only prerelease
versions available, a specific version is requested,
or the prerelease version is found in a reasonable source (active
environment, explicit path, etc. but not `PATH`).
For example, `uv python install 3.13 && uv run python --version` will no
longer use `3.13.0rc2` unless that is the only Python version available,
`--python 3.13` is used, or that's the Python version that is present in
`.venv`.
## Summary
Add maturin build flag to set 64kb page size on PPC64 and PPC64LE
architectures. Not aware of modern systems that use 4kb pages.
Resolves#6528
## Test Plan
ppc64le gnu dynamic-linked and musl static-linked binary builds were
tested successfully on an IBM Power9 system running RHEL 8.8. I do not
have access to other types of PPC64 systems for testing.
## Summary
Closes https://github.com/astral-sh/uv/issues/7030
This removes our custom `ENTRYPOINT` just for the additional docker
tags, and makes it empty (to avoid possible upstream surprises if any)
and moves running uv to `CMD` for consistency.
This approach is probably the in-between solution from the discussion in
https://github.com/astral-sh/uv/issues/7030#issuecomment-2329443719 and
would work for everyone's use cases.
## Test Plan
Tested release workflow in
https://github.com/samypr100/uv/actions/runs/10711049920
The default CMD still gives a nice default.
```shell
> docker run ghcr.io/samypr100/uv:0.4.5-alpine
An extremely fast Python package manager.
Usage: uv [OPTIONS] <COMMAND>
Commands:
run Run a command or script
init Create a new project
add Add dependencies to the project
remove Remove dependencies from the project
sync Update the project's environment
lock Update the project's lockfile
export Export the project's lockfile to an alternate format
tree Display the project's dependency tree
tool Run and install commands provided by Python packages
python Manage Python versions and installations
pip Manage Python packages with a pip-compatible interface
venv Create a virtual environment
build Build Python packages into source distributions and wheels
cache Manage uv's cache
version Display uv's version
help Display documentation for a command
```
## Summary
Closes https://github.com/astral-sh/uv/issues/5610
This PR introduces additional images with the uv/uvx binaries from
scratch for both amd64/arm64 and make the mapping easy to configure by
generating the Dockerfile on the fly. This approach focuses on
minimizing CI time by taking advantage of dedicating a worker per
mapping (20-30s~ per job).
This PR also fixes `org.opencontainers.image.version` for all tags
(including the one from `scratch) to contain the right release version
instead of branch name `main` (default when no tag patterns are
specified).
For example, on release `x.y.z`, this will publish the following image
tags with format `ghcr.io/astral-sh/uv:{tag}` with manifests for both
amd64/arm64. This also include `x.y` tags for each respective additional
tag.
* From **scratch**: `latest`, `x.y.z`, `x.y` (currently being published)
* From **alpine:3.20**: `alpine`, `alpine3.20`, `x.y.z-alpine`,
`x.y.z-alpine3.20`
* From **debian:bookworm-slim**: `debian-slim`, `bookworm-slim`,
`x.y.z-debian-slim`, `x.y.z-bookworm-slim`
* From **buildpack-deps:bookworm**: `debian`, `bookworm`,
`x.y.z-debian`, `x.y.z-bookworm`
* From **python:3.12-alpine**: `python3.12-alpine`,
`x.y.z-python3.12-alpine`
* From **python:3.11-alpine**: `python3.11-alpine`,
`x.y.z-python3.11-alpine`
* From **python:3.10-alpine**: `python3.10-alpine`,
`x.y.z-python3.10-alpine`
* From **python:3.9-alpine**: `python3.9-alpine`,
`x.y.z-python3.9-alpine`
* From **python:3.8-alpine**: `python3.8-alpine`,
`x.y.z-python3.8-alpine`
* From **python:3.12-bookworm**: `python3.12-bookworm`,
`x.y.z-python3.12-bookworm`
* From **python:3.11-bookworm**: `python3.11-bookworm`,
`x.y.z-python3.11-bookworm`
* From **python:3.10-bookworm**: `python3.10-bookworm`,
`x.y.z-python3.10-bookworm`
* From **python:3.9-bookworm**: `python3.9-bookworm`,
`x.y.z-python3.9-bookworm`
* From **python:3.8-bookworm**: `python3.8-bookworm`,
`x.y.z-python3.8-bookworm`
* From **python:3.12-slim-bookworm**: `python3.12-slim-bookworm`,
`x.y.z-python3.12-slim-bookworm`
* From **python:3.11-slim-bookworm**: `python3.11-slim-bookworm`,
`x.y.z-python3.11-slim-bookworm`
* From **python:3.10-slim-bookworm**: `python3.10-slim-bookworm`,
`x.y.z-python3.10-slim-bookworm`
* From **python:3.9-slim-bookworm**: `python3.9-slim-bookworm`,
`x.y.z-python3.9-slim-bookworm`
* From **python:3.8-slim-bookworm**: `python3.8-slim-bookworm`,
`x.y.z-python3.8-slim-bookworm`
## Summary
Noticed that running `cargo dev generate-all` on `main` produced changes
and saw that that the command is not run on the CI nor as a pre-commit
hook.
Not sure if having the command running as a pre-commit hook is something
we want, so I can remove it if you prefer. I find that nice to have as
it's probably easy to forget to run it, especially for new contributors
(and it will only run if there are changes in `uv_cli` or `uv_settings`
crates).
## Test Plan
- Added `cargo dev generate-all --mode check` on the CI, which produced
[this failing
job](https://github.com/astral-sh/uv/actions/runs/10648055597/job/29516699393)
- Ran `cargo dev generate-all` locally and committed the changes, which
produced [this succeeding
job](https://github.com/astral-sh/uv/actions/runs/10648076910/job/29516744942)
As suggested by @samypr100 on #6680:
https://github.com/astral-sh/uv/pull/6680#issuecomment-2313607984
## Summary
Instead of using `UV_INTERNAL__TEST_DIR`, it simply exports `TEMP` when
running Windows jobs.
## Test Plan
I'm going to run this manually under ProcMon on my Windows machine and
see where uv writes temp files, hopefully to the dev drive and not
`%(LOCAL)APPDATA%` or something.
I'm going to commit a dummy code change and look at build time changes
in CI.
## Summary
Closes https://github.com/astral-sh/uv/issues/6678
This change would publish an additional tag that includes only
`major.minor`.
For a release with x.y.z, this would publish the tags:
* `ghcr.io/astral-sh/uv:latest`
* `ghcr.io/astral-sh/uv:x.y.z`
* `ghcr.io/astral-sh/uv:x.y`
## Summary
This PR makes `cargo test | windows` faster in CI.
### Before
### After
## Also
This PR disables the `brotli` feature of `async-compression` since it's
not strictly needed, but this has little to do with the improvements
(it's still less code to build).
This PR introduces additional code in uv tool uninstall to ignore errors
(that only seem to happen on ReFS, ie. on Dev Drives) akin to "the thing
we're trying to delete cannot be deleted because it's already being
deleted".
If `raw_os_error` was stable we could do u32 matching instead of that
`.to_string().contains()` abomination.
## Summary
This PR parallelizes multi-platform builds using multiple workers (hence
the new docker-build / docker-publish jobs), this seems to save about ~8
minutes.
This is partial work extracted from
https://github.com/astral-sh/uv/pull/6053 than is standalone
From 8 to 16 cores, 32 to 64 GB ram. Testing on Windows first because
it's the bottleneck.
Previously tested in #2515 to no effect, maybe better now that we have a
development drive?
## Summary
This is an experimental PR to replace more unsafe calls with more rust
while still trying to keep the binary size small enough. These changes
roughly increase the size of the trampolines to about 40kb~. This is a
alternate PR to https://github.com/astral-sh/uv/pull/5751.
The primary changes here include
* Switch to use rust path components for ease of path management
* Leverage `std::process::exit` for process exit and cleanup
* Use `std::io::Error::last_os_error` for IO Errors to remove
`FormatMessage` complexity
* Use `std::env::current_exe` to get the current executable instead of
`GetModuleFileNameA`
## Test Plan
Added one more existing test case to trampoline tests.
Still need to verify dunce::canonicalize is desired or not on
find_python_exe.
---------
Co-authored-by: konstin <konstin@mailbox.org>
To enforce the 100 character line limit in markdown files introduced in
https://github.com/astral-sh/uv/pull/5635, and to automate the
formatting of markdown files, i've added prettier and formatted our
markdown files with it.
I've excluded the changelog and the generated references documentation
from this for having too many changes, but we can also include them.
I'm not particular on which style we use. My main motivations are
(major) not having to reflow markdown files myself anymore and (minor)
consistence between all markdown files. I've chosen prettier for similar
reason as we chose black, it's a single good style that's automated and
shared in the community. I do prefer prettier's style of not breaking
inside of a link name though.
This PR is in two parts, the first adds prettier to CI and documents
using it, while the second actually formats the docs. When merge
conflicts arise, we can drop the last commit and regenerate it with `npx
prettier --prose-wrap always --write BENCHMARKS.md CONTRIBUTING.md
README.md STYLE.md docs/*.md docs/concepts/**/*.md docs/guides/**/*.md
docs/pip/**/*.md`.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Part of https://github.com/astral-sh/uv/issues/5713
Shaves 50s or ~25% off the Ubuntu test run. Maybe 30s or 8% off macOS.
Windows already uses the GitHub distributions.
Note this is some of our only test coverage for Python version installs,
we may want to add separate coverage to compensate.
## Summary
This PR deprecates the `--isolated` flag. The treatment varies across
the APIs:
- For non-preview APIs, we warn but treat it as equivalent to
`--no-config`.
- For preview APIs, we warn and ignore it, with two exceptions...
- For `tool run` and `run` specifically, we don't even warn, because we
can't differentiate the command-specific `--isolated` from the global
`--isolated`.
Add the tests added in #5204 to CI. The crate is not part of the
workspace (it requires nightly) and is windows only, so we have to test
it separately.
---------
Co-authored-by: samypr100 <3933065+samypr100@users.noreply.github.com>
## Summary
After #5337, `fetch-download-metadata.py` fetches not just from
`python-build-standalone`, so updates the workflow to
`sync-python-releases.yml`.
Also includes `crates/uv-python/download-metadata.json` in `add-paths`.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Currently, `uv` refuses to install anything on GraalPy. This is
currently blocking GraalPy testing with cibuildwheel, since manylinux
includes both `uv` and `graalpy` (but doesn't test with `uv`), whereas
cibuildwheel defaults to `uv`. See e.g.
https://github.com/pypa/cibuildwheel/actions/runs/9956369360/job/27506182952?pr=1538
where it gives
```
+ python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
* Creating isolated environment: venv+uv...
* Using external uv from /usr/local/bin/uv
* Installing packages in isolated environment:
- setuptools >= 40.8.0
> /usr/local/bin/uv pip install "setuptools >= 40.8.0"
< error: Unknown implementation: `graalpy`
```
## Test Plan
I simply based the GraalPy support on PyPy and added some small tests.
I'm open to discussing how to test this. GraalPy is available for
manylinux images and with setup-python, so we should be able to add
tests against it to the CI. I locally confirmed by installing `uv` into
a GraalPy venv and then trying things like `uv pip install Pillow` and
testing those extensions.
## Summary
This approach is based on https://github.com/PrefectHQ/docs. Rather than
publishing docs in the uv repo, we push to an independent repo that's
used solely to house the docs. In Prefect's case, this allows them to
publish versioned documentation (we should do that too). For us, though,
the benefit is that we can publish the Ruff and uv docs as a single site
(docs.astral.sh).
Prefect clones the source repo and builds the documentation from the
`docs` repo (i.e., the action runs in the `docs` repo). In our case,
I've instead set it up such that the action runs in `uv` (and later in
`ruff` too), clones the `docs` repo, and puts up a PR in that separate
repo. Because of these requirements, we have to use a PAT rather than a
deploy key (as PATs cannot do GitHub-specific things like create PRs --
they can only operate over the Git CLI).
See: https://github.com/astral-sh/docs/pull/2.
## Summary
We want to have consistency between the Ruff and uv documentation for
the upcoming release. We don't love the Ruff docs, but we'd rather have
consistency and then work towards improving them both, rather than have
two very-different documentation sites that both have weaknesses.
The setup here is simpler than in Ruff as: (1) we don't yet generate any
docs from Rust and (2) we don't try to reuse the README in the uv
documentation (which adds a lot of complexity in Ruff). So the change
here is mostly a 1-to-1 port to MkDocs.
## Test Plan
## Summary
This PR adds a new job to test that PyPy venvs are correctly created on
Windows and they contain all the expected binaries.
## Test Plan
Just let CI run.
## Summary
Should fix#2092.
This PR changes `uv venv` so it also creates symlinks to `pypy` on Unix
and copies executables on Windows when creating a new environment using
PyPy.
I found a bit of discrepancy between creation of a venv using `python`
and `uv`, as using `python` brings all the executables with it. While
`uv` brings only those without any version number, at least on Windows.
The behaviour is different on Unix as we take the versioned symlinks
too.
Some examples below.
`python -m venv` generates the following `Scripts` folder.
```
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/14/2024 15:41 2031 activate
-a---- 7/14/2024 15:41 1029 activate.bat
-a---- 7/14/2024 15:41 9033 Activate.ps1
-a---- 7/14/2024 15:41 393 deactivate.bat
-a---- 7/14/2024 15:40 27648 libffi-8.dll
-a---- 7/14/2024 15:41 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 15:41 108424 pip.exe
-a---- 7/14/2024 15:41 108424 pip3.10.exe
-a---- 7/14/2024 15:41 108424 pip3.exe
-a---- 7/14/2024 15:41 79360 pypy.exe
-a---- 7/14/2024 15:41 79360 pypy3.10.exe
-a---- 7/14/2024 15:41 79360 pypy3.10w.exe
-a---- 7/14/2024 15:41 79360 pypy3.exe
-a---- 7/14/2024 15:41 79360 pypyw.exe
-a---- 7/14/2024 15:41 79360 python.exe
-a---- 7/14/2024 15:41 79360 python3.10.exe
-a---- 7/14/2024 15:41 79360 python3.exe
-a---- 7/14/2024 15:41 79360 pythonw.exe
```
`uv venv` instead generates this.
```
-a---- 7/14/2024 16:27 3360 activate
-a---- 7/14/2024 16:27 2251 activate.bat
-a---- 7/14/2024 16:27 2627 activate.csh
-a---- 7/14/2024 16:27 4191 activate.fish
-a---- 7/14/2024 16:27 3875 activate.nu
-a---- 7/14/2024 16:27 2766 activate.ps1
-a---- 7/14/2024 16:27 2378 activate_this.py
-a---- 7/14/2024 16:27 1728 deactivate.bat
-a---- 7/13/2024 19:19 27648 libffi-8.dll
-a---- 7/13/2024 19:19 44290560 libpypy3.10-c.dll
-a---- 7/14/2024 16:27 1215 pydoc.bat
-a---- 7/13/2024 19:19 79360 pypy.exe
-a---- 7/13/2024 19:19 79360 pypyw.exe
-a---- 7/13/2024 19:19 79360 python.exe
-a---- 7/13/2024 19:19 79360 pythonw.exe
```
## Test Plan
To verify the correct behaviour:
1. Download and install PyPy from [official
website](https://www.pypy.org/download.html)
2. Call `uv venv -p <path_to_pypy_>`
3. Run `.\.venv\Scripts\activate` on Windows or
`./.venv/Scripts/activate` on Unix
4. Run `pypy`
I thought of writing some automated tests but I couldn't rely on `uv
python install` command to install PyPy as it's not in the list of
installable Python builds.
## Summary
Move completely off tokio's multi-threaded runtime. We've slowly been
making changes to be smarter about scheduling in various places instead
of depending on tokio's general purpose work-stealing, notably
https://github.com/astral-sh/uv/pull/3627 and
https://github.com/astral-sh/uv/pull/4004. We now no longer benefit from
the multi-threaded runtime, as we run on all I/O on the main thread.
There's one remaining instance of `block_in_place` that can be swapped
for `rayon::spawn`.
This change is a small performance improvement due to removing some
unnecessary overhead of the multi-threaded runtime (e.g. spawning
threads), but nothing major. It also removes some noise from profiles.
## Test Plan
```
Benchmark 1: ./target/profiling/uv (resolve-warm)
Time (mean ± σ): 14.9 ms ± 0.3 ms [User: 3.0 ms, System: 17.3 ms]
Range (min … max): 14.1 ms … 15.8 ms 169 runs
Benchmark 2: ./target/profiling/baseline (resolve-warm)
Time (mean ± σ): 16.1 ms ± 0.3 ms [User: 3.9 ms, System: 18.7 ms]
Range (min … max): 15.1 ms … 17.3 ms 162 runs
Summary
./target/profiling/uv (resolve-warm) ran
1.08 ± 0.03 times faster than ./target/profiling/baseline (resolve-warm)
```
## Summary
Partially closes#1917
This PR picks up on some of the great work from #1864 and opted to keep
`panic_immediate_abort` (for size reasons). I split the PR in different
isolated commits in case we want to separate/cherry-pick them out.
1. The first commit ports mostly all std changes from that PR into this
PR. Binary sizes stayed the same ~16kb.
2. The second commit migrates our existing usage of windows-sys to
windows for a safer ffi calls with Results!. It also changes all large
unsafe blocks to be isolated to the actual unsafe calls, and switches
some areas to use std such as getenv port ( which seemed buggy! ) from
launcher.c. In addition, this also adds more error checking in order to
match some missing assertions from distlib's launcher.c. Note, due to
the additional .text data, the binary sizes increased to ~20.5kb, but we
can cut back on some of the added error msgs as needed.
3. The third commit switches to using xwin for building on all 3
supported trampoline targets for sanity, and adds a CI bloat check for
core::fmt and panic as a precaution. Sadly, this will invalidate the
xwin cache on the first run.
## Test Plan
Most changes were tested on a couple of local GUI apps and console apps,
also tested some of the error states manually by using SetLastError at
different points in the code and/or passing in invalid handles.
I'm not sure how far we can get with migrating some of the other calls
without increasing binary size substantially. An initial attempt at
using std::path didn't seem so bad size wise when I tried it (~1k). On
other cases, such as std::process::exit added ~10k to the total binary
size.
---------
Co-authored-by: konstin <konstin@mailbox.org>
Whew this is a lot.
The user-facing changes are:
- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"
The internal changes are:
- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
## Summary
Updates default toolchain to
[1.79](https://blog.rust-lang.org/2024/06/13/Rust-1.79.0.html).
There didn't seem to be any breaking changes besides xwin now requiring
lld llvm linker.
## Test Plan
Existing tests.
## Summary
See: https://github.com/axodotdev/cargo-dist/releases/tag/v0.17.0.
Relevant:
> The only reason you might want to override this setting is if you're
using [dispatch-releases =
true](https://opensource.axo.dev/cargo-dist/book/reference/config.html#dispatch-releases)
and you really want your git tag to be the last operation in your
release process (because creating a GitHub Release necessarily creates
the git tag if it doesn't yet exist, and many organizations really don't
like when you delete/change git tags). In this case setting
github-release = "announce" will accomplish that, but the above race
conditions would then apply.
We _do_ use `dispatch-releases = true`, and we _do_ want the git tag to
be the last operation, so we need to set `github-release = "announce"`
to preserve our current behavior.
So we can skip them when there are not code changes and still enforce
our required checks (xref #4438)
Otherwise, the names are dynamic and they are forever expected (see
#4426 for example)
It was becoming problematic that the virtual environment test context
diverged from the other one i.e. we had to implement filtering twice.
This combines the contexts and tweaks the `TestContext` API and
filtering mechanisms for Python versions. Combined with my previous
changes to the test context at #4364 and
https://github.com/astral-sh/uv/pull/4368 this finally unblocks the
snapshots for test cases in #4360 and
https://github.com/astral-sh/uv/pull/4362.
This PR re-adds the `aarch64-unknown-linux-gnu` binary, which will also
add the `manylinux_2_28` wheel for `aarch64` (in addition to the now
dual-tagged `musllinux_1_1` and `manylinux_2_217` wheel for `aarch64`).
We can consider dropping that _wheel_, but in my assessment removing a
release asset should now be treated as a breaking change -- so removing
it in a patch release was incorrect.
Closes https://github.com/astral-sh/uv/issues/4122.
Extends #3726
Moves toolchain storage out of `UV_BOOTSTRAP_DIR` (`./bin`) into the
proper user data directory as defined by #3726.
Replaces `UV_BOOTSTRAP_DIR` with `UV_TOOLCHAIN_DIR` for customization.
Installed toolchains will be discovered without opt-in, but the idea is
still that these are not yet user-facing.
## Summary
This seems to be one of the most consistent benchmark cases we have in
terms of standard deviation:
```
➜ hyperfine "target/profiling/main pip compile scripts/requirements/airflow.in" --runs 200
Benchmark 1: target/profiling/main pip compile scripts/requirements/airflow.in
Time (mean ± σ): 292.6 ms ± 6.6 ms [User: 414.1 ms, System: 194.2 ms]
Range (min … max): 282.7 ms … 320.1 ms 200 runs
```
For smaller benchmarks, scispacy and dtlssocket seem to be a bit more
consistent than our current jupyter benchmark, but it hasn't given us
any problems so I'll leave it for now.
Updates our Python interpreter discovery to conform to the rules
described in #2386, please see that issue for a full description of the
behavior. Briefly, we now will search for interpreters that satisfy a
requested version without stopping at the first Python executable.
Additionally, if retrieving information about an interpreter fails we
will continue to search for a working interpreter. We also add the
plumbing necessary to request Python implementations other than CPython,
though we do not add support for other implementations at this time.
A major internal goal of this work is to prepare for user-facing managed
toolchains i.e. fetching a requested version during `uv run`. These APIs
are not introduced, but there is some managed toolchain handling as
required for our test suite.
Some noteworthy implementation changes:
- The `uv_interpreter::find_python` module has been removed in favor of
a `uv_interpreter::discovery` module.
- There are new types to help structure interpreter requests and track
sources
- Executable discovery is implemented as a big lazy iterator and is a
central authority for source precedence
- `uv_interpreter::Error` variants were split into scoped types in each
module
- There's much more unit test coverage, but not for Windows yet
Remaining work:
- [x] Write new test cases
- [x] Determine correct behavior around executables in the current
directory
- _Future_: Combine `PythonVersion` and `VersionRequest`
- _Future_: Consider splitting `ManagedToolchain` into local and remote
variants
- _Future_: Add Windows unit test coverage
- _Future_: Explore behavior around implementation precedence (i.e.
CPython over PyPy)
Refactors split into:
- #3329
- #3330
- #3331
- #3332Closes#2386
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Followup to #3624. Drops the native manylinux wheel in favor of the
statically linked musl one with a manylinux2014 tag.
<!-- How was it tested? -->
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
## Summary
Restore API-compatibility with pre-1.1.0 versions of the `zip` crate,
and pin the dependency to the 0.6 series, due to concerns discussed in
https://github.com/astral-sh/uv/issues/3642.
## Test Plan
```
cargo run -p uv-dev -- fetch-python
cargo test
```
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Fix#3439 by retagging the statically linked musl binary for
manylinux2014. This is the most conservative option;
the regular manylinux one could be removed as well, and I expect other
archs could probably do this as well.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
Testing on manylinux.
<!-- How was it tested? -->
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
This is needed to include uv in manylinux, which has both the
musllinux_1_1 and musllinux_1_2 images.
## Summary
We are looking into including uv in the `pypa/manylinux` docker images,
so that cibuildwheel will be able to use it. But to do so, we need all
actively supported images to be able to take it, including
musllinux_1_1.
## Test Plan
The test was updated to make sure Alpine 3.12, which musllinux was based
on, can run uv.
The binary build does need to be triggered on this PR to make sure it
works. Ah,
5b6040184d/.github/workflows/build-binaries.yml (L15-L20)
is a good idea, nice.
Will need a followup to fill out the other platforms `pypa/manylinux`
supports for musllinux.
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Separates Windows tests from the rest because it's a pain and drops the
`python-patch` feature from testing so we can use the GitHub Actions
Python versions which bootstrap in 0 seconds instead of 2 minutes.
## Summary
This PR adds system install tests to verify the behavior described in
#2798. It turns out this behavior _also_ affects Fedora and Amazon
Linux, we just didn't have the right conditions enabled (specifically,
you need to create the virtualenv with `python -m venv` to get these
symlinks), so the test suite was expanded to capture that.
The issue itself is also fixed by way of deduplicating the
`site-packages` entries.
Closes: https://github.com/astral-sh/uv/issues/2798
See https://github.com/astral-sh/uv/issues/2617
Note this also includes:
- #2918
- #2931 (pending)
A first step towards Python toolchain management in Rust.
First, we add a new crate to manage Python download metadata:
- Adds a new `uv-toolchain` crate
- Adds Rust structs for Python version download metadata
- Duplicates the script which downloads Python version metadata
- Adds a script to generate Rust code from the JSON metadata
- Adds a utility to download and extract the Python version
I explored some alternatives like a build script using things like
`serde` and `uneval` to automatically construct the code from our
structs but deemed it to heavy. Unlike Rye, I don't generate the Rust
directly from the web requests and have an intermediate JSON layer to
speed up iteration on the Rust types.
Next, we add add a `uv-dev` command `fetch-python` to download Python
versions per the bootstrapping script.
- Downloads a requested version or reads from `.python-versions`
- Extracts to `UV_BOOTSTRAP_DIR`
- Links executables for path extension
This command is not really intended to be user facing, but it's a good
PoC for the `uv-toolchain` API. Hash checking (via the sha256) isn't
implemented yet, we can do that in a follow-up.
Finally, we remove the `scripts/bootstrap` directory, update CI to use
the new command, and update the CONTRIBUTING docs.
<img width="1023" alt="Screenshot 2024-04-08 at 17 12 15"
src="https://github.com/astral-sh/uv/assets/2586601/57bd3cf1-7477-4bb8-a8e9-802a00d772cb">
Reproduced https://github.com/astral-sh/uv/issues/2941 and confirmed
fix.
We probably ought to have some ecosystem test coverage — this seems like
a good starting point we can extend to other projects in the future.
Following #2735 adds a system check that uses Homebrew. I think we were
never were actually using Homebrew's Python in the past, we were mislead
or something changed in the runners recently that broke it.
## Summary
Ensures that (e.g.) installs from conda-forge, Homebrew, and other
distributions don't expose `uv self update` at all.
We'll still show `uv self update` for `pip install uv`, but it will fail
with a good error. Removing the `uv self update` from `pip`-installed
`uv` is more complicated, since we'd need to build separately for the
installer vs. for PyPI.
Closes#2588.
Add a single job for for fast lint tools. Rustfmt for rust, ruff for
python formatting and linting, prettier avoids inconsistent formatter
changes between pycharm and vscode.
If you look at https://pypi.org/project/uv/0.1.22/#files...
- We didn't upload the ARMv6 wheel (I thought I had removed the `# Skip
for `arm`, which is not supported by PyPI.`), it must've gotten re-added
in a rebase or something.
- We lost the musllinux builds for ARM. I think this is because I built
them as manylinux.
Installing and importing numpy tests for two cases:
* The python architecture and the package architecture don't match
(https://github.com/astral-sh/uv/issues/2326)
* The libc of python and that of the package don't match on linux
(musllinux vs manylinux, picking a compatible manylinux version)
All pylint deps are py3-none-any, so they don't catch those cases.
Giving this a try... just making all of these a normal part of CI.
This is probably slightly slower than our normal CI, but not by much (it
depends how bad of a roll we get on the Windows network performance).
Includes #2309 to reduce the overhead of adding more platforms.
Alternatively, we could gate these with a label and just run on main by
default (i.e. #2308)
## Summary
Adjust maturing version to use env var for reusability in the workflow.
## Test Plan
CI `Build binaries` workflow uses correct version and passes
## Summary
`pip` uses `sysconfig` for Python 3.10 and later by default; however, it
falls back to `distutils` for earlier Python versions, and distros can
actually tell `pip` to continue falling back to `distutils` via the
`_PIP_USE_SYSCONFIG` variable.
By _always_ using `sysconfig`, we're doing the wrong then when
installing into some system Pythons, e.g., on Debian prior to Python
3.10.
This PR modifies our logic to mirror `pip` exactly, which is what's been
recommended to me as the right thing to do.
Closes https://github.com/astral-sh/uv/issues/2113.
## Test Plan
Most notably, the new Debian tests pass here (which fail on main:
https://github.com/astral-sh/uv/pull/2144).
I also added Pyston as a second stress-test.
`uv --system` is failing in GitHub Actions, because `py --list-paths`
returns all the pre-cached Pythons:
```
-V:3.12 * C:\hostedtoolcache\windows\Python\3.12.2\x64\python.exe
-V:3.12-32 C:\hostedtoolcache\windows\Python\3.12.2\x86\python.exe
-V:3.11 C:\hostedtoolcache\windows\Python\3.11.8\x64\python.exe
-V:3.11-32 C:\hostedtoolcache\windows\Python\3.11.8\x86\python.exe
-V:3.10 C:\hostedtoolcache\windows\Python\3.10.11\x64\python.exe
-V:3.10-32 C:\hostedtoolcache\windows\Python\3.10.11\x86\python.exe
-V:3.9 C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe
-V:3.9-32 C:\hostedtoolcache\windows\Python\3.9.13\x86\python.exe
-V:3.8 C:\hostedtoolcache\windows\Python\3.8.10\x64\python.exe
-V:3.8-32 C:\hostedtoolcache\windows\Python\3.8.10\x86\python.exe
-V:3.7 C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
-V:3.7-32 C:\hostedtoolcache\windows\Python\3.7.9\x86\python.exe
```
So, our default selector returns the first entry here. But none of these
are actually in `PATH` except the one that the user installed via
`actions/setup-python@v5` -- that's the point of the action, that it
puts the correct versions in `PATH`.
It seems to me like we should prioritize `PATH` over `py --list-paths`.
Is there a good reason not to do this?
Closes: https://github.com/astral-sh/uv/issues/2056
## Summary
This is essentially a wrapper around something like `--python $(which
python3)`, but gives users a portable and streamlined way to solve the
common pain point of using `uv` in GitHub Actions or a Docker container.
See: https://github.com/astral-sh/uv/issues/1526.
## Summary
Closes#1943
Makes sure `build-binaries` and `publish-pypi` workflows are compatible
with `actions/{download,upload}-artifact@v4`. In nature, this PR is very
similar to the changes in https://github.com/astral-sh/ruff/pull/10105.
This PR also updates cargo-dist.
## Test Plan
I ran a small non-dry-run [smoke
test](https://github.com/samypr100/uv/actions/runs/8027864059) on my own
fork CI with only linux builds (for speed) and those jobs seem to work
at a glance.
Charlie Marsh
Zanie Blue
konsti
Jo
renovate[bot]
Mathieu Kniewallner
Hamir Mahal
samypr100
Tim Felgentreff
sobolevn
Aarni Koskela
Bas Schoenmaeckers
bluss
Tom
Ibraheem Ahmed
Amos Wenger
Andrew Gallant
Tim Felgentreff
Silvano Cerza
Henry Schreiner
Ben Beasley
Alex Waygood
Zander
Micha Reiser