Part of https://github.com/astral-sh/uv/issues/4392
We shouldn't link to PyPI, and dropping the workspace-level
documentation link should mean that we get the auto-generated `docs.rs`
links.
* feat: reject ZIP archives with improbable filenames
Signed-off-by: William Woodruff <william@astral.sh>
* use my PR for async_zip temporarily
Signed-off-by: William Woodruff <william@astral.sh>
* update snapshot
Signed-off-by: William Woodruff <william@astral.sh>
* two more tests
Signed-off-by: William Woodruff <william@astral.sh>
* update rev
Signed-off-by: William Woodruff <william@astral.sh>
---------
Signed-off-by: William Woodruff <william@astral.sh>
## Summary
This PR allows pyx to send down hashes for zstandard-compressed
tarballs. If the hash is present, then the file is assumed to be present
at `${wheel_url}.tar.zst`, similar in design to PEP 658
`${wheel_metadata}.metadata` files. The intent here is that the index
must include the wheel (to support all clients and support
random-access), but can optionally include a zstandard-compressed
version alongside it.
## Summary
There isn't any risk here, and we have reports of at least one zip file
with more than one (but fewer than, e.g., 10) null bytes.
Closes https://github.com/astral-sh/uv/issues/15451.
Venvs should not be in source distributions, and on Unix, we now reject
them for having a link outside the source directory. This PR adds a hint
for that since users were confused (#15096).
In the process, we're differentiating IO errors for format error for
uncompression generally.
Fixes#15096
## Summary
uv will now reject ZIP files that meet any of the following conditions:
- Multiple local header entries exist for the same file with different
contents.
- A local header entry exists for a file that isn't included in the
end-of-central directory record.
- An entry exists in the end-of-central directory record that does not
have a corresponding local header.
- The ZIP file contains contents after the first end-of-central
directory record.
- The CRC32 doesn't match between the local file header and the
end-of-central directory record.
- The compressed size doesn't match between the local file header and
the end-of-central directory record.
- The uncompressed size doesn't match between the local file header and
the end-of-central directory record.
- The reported central directory offset (in the end-of-central-directory
header) does not match the actual offset.
- The reported ZIP64 end of central directory locator offset does not
match the actual offset.
We also validate the above for files with data descriptors, which we
previously ignored.
Wheels from the most recent releases of the top 15,000 packages on PyPI
have been confirmed to pass these checks, and PyPI will also reject ZIPs
under many of the same conditions (at upload time) in the future.
In rare cases, this validation can be disabled by setting
`UV_INSECURE_NO_ZIP_VALIDATION=1`. Any validations should be reported to
the uv issue tracker and to the upstream package maintainer.
## Summary
Make the use of `Self` consistent. Mostly done by running `cargo clippy
--fix -- -A clippy::all -W clippy::use_self`.
## Test Plan
<!-- How was it tested? -->
No need.
## Summary
Closes#12163.
## Test Plan
Created an offending source distribution with this script:
```python
import io
import tarfile
import textwrap
import time
PKG_NAME = "badpkg"
VERSION = "0.1"
DIST_NAME = f"{PKG_NAME}-{VERSION}"
ARCHIVE = f"{DIST_NAME}.tar.gz"
def _bytes(data: str) -> io.BytesIO:
"""Helper: wrap a text blob as a BytesIO for tarfile.addfile()."""
return io.BytesIO(data.encode())
def main(out_path: str = ARCHIVE) -> None:
now = int(time.time())
with tarfile.open(out_path, mode="w:gz") as tar:
def add_file(path: str, data: str, mode: int = 0o644) -> None:
"""Add a regular file whose *content* is supplied as a string."""
buf = _bytes(data)
info = tarfile.TarInfo(path)
info.size = len(buf.getbuffer())
info.mtime = now
info.mode = mode
tar.addfile(info, buf)
# ── top‑level setup.py ───────────────────────────────────────────────
setup_py = textwrap.dedent(f"""\
from setuptools import setup, find_packages
setup(
name="{PKG_NAME}",
version="{VERSION}",
packages=find_packages(),
)
""")
add_file(f"{DIST_NAME}/setup.py", setup_py)
# ── minimal package code ─────────────────────────────────────────────
add_file(f"{DIST_NAME}/{PKG_NAME}/__init__.py", "# placeholder\\n")
# ── the malicious symlink ────────────────────────────────────────────
link = tarfile.TarInfo(f"{DIST_NAME}/{PKG_NAME}/evil_link")
link.type = tarfile.SYMTYPE
link.mtime = now
link.mode = 0o777
link.linkname = "../../../outside.txt"
tar.addfile(link)
print(f"Created {out_path}")
if __name__ == "__main__":
main()
```
Verified that both `pip install` and `uv pip install` rejected it.
I also changed `link.linkname = "../../../outside.txt"` to
`link.linkname = "/etc/outside"`, and verified that the absolute path
was rejected too.
## Summary
We mapped both `.tgz` and `.tar.gz` to the same enum variant; later,
though, we made the assumption that a file marked with that variant
ended with exactly `.tar.gz`. Instead, we need to preserve the
originating suffix.
Closes https://github.com/astral-sh/uv/issues/13372.
#5577 fixed a bug on macos due to dynamically linking lzma/xz through
static linking. In #7686, this feature was moved to the performance
category.
This PR moves the `xz2/static` back to the general default features,
and, inspired by https://github.com/Homebrew/homebrew-core/pull/222211,
it structures and documents the feature flags cleaner.
We need to take care that this feature does not accidentally disable
features we want.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Fixes#12618
Instead of succeeding the user now gets:
```
uvdloc pip install osqp==1.0.2 --reinstall --python-platform=linux
Resolved 7 packages in 171ms
× Failed to download `osqp==1.0.2`
├─▶ Failed to extract archive
╰─▶ a computed CRC32 value did not match the expected value
```
I am not entirely sure if we have infra for testing this kind of thing,
but it would be nice to check in a test or two. I'm also not entirely
clear if there's any cases where these checks are overzealous.
## Summary
A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.
IIUC, this _does_ require bumping our MSRV.
When performing a noop sync, we don't need the rayon threadpool, yet we
pay for its initialization:
Be making the initialization lazy, we avoid that cost:
This code runs every time before user code in `uv run`.
This means that before calling rayon, one now needs to call
`LazyLock::force(&RAYON_INITIALIZE);`.
Performance mode (CPU 0 is a perf core):
```
$ taskset -c 0 hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
Time (mean ± σ): 4.5 ms ± 0.1 ms [User: 2.7 ms, System: 1.8 ms]
Range (min … max): 4.4 ms … 6.4 ms 640 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
Time (mean ± σ): 4.4 ms ± 0.1 ms [User: 2.7 ms, System: 1.6 ms]
Range (min … max): 4.3 ms … 5.0 ms 679 runs
Summary
/home/konsti/projects/uv/target/profiling/uv sync ran
1.03 ± 0.04 times faster than /home/konsti/projects/uv/uv-main sync
```
Power saver mode:
```
$ hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
Time (mean ± σ): 28.1 ms ± 1.2 ms [User: 15.5 ms, System: 20.3 ms]
Range (min … max): 25.7 ms … 31.9 ms 102 runs
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
Time (mean ± σ): 24.0 ms ± 1.2 ms [User: 13.8 ms, System: 9.9 ms]
Range (min … max): 22.2 ms … 28.2 ms 122 runs
Summary
/home/konsti/projects/uv/target/profiling/uv sync ran
1.17 ± 0.08 times faster than /home/konsti/projects/uv/uv-main sync
```
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html
Before that, there were 91 separate integration tests binary.
(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
closes#7365
Summary
This pull request adds support for additional file extension aliases in
the SourceDistExtension and ExtensionError enums. The newly supported
file extensions include .tbz, .tgz, .txz, .tar.lz, .tar.lzma. These
changes align the extensions supported by the SourceDistExtension with
those used in Python packaging tools, enhancing compatibility with a
broader range of source distribution formats.
Test Plan
should be added or updated to verify that the new extensions are
correctly recognized as valid source distributions and that errors are
correctly raised when unsupported extensions are provided.
## Summary
Replace the unmaintained `tokio-tar` crate with the `krata-tokio-tar`
fork. The latter just merged a fix necessary for the crate to work on
PowerPC, and has better chances of future maintenance.
Fixes#3423
## Test Plan
`cargo test`
This is preparatory work for the upload functionality, which needs to
read the METADATA file and attach its parsed contents to the POST
request: We move finding the `.dist-info` from `install-wheel-rs` and
`uv-client` to a new `uv-metadata` crate, so it can be shared with the
publish crate.
I don't properly know if its the right place since the upload code isn't
ready, but i'm PR-ing it now because it already had merge conflicts.
## Summary
This PR adds a `DistExtension` field to some of our distribution types,
which requires that we validate that the file type is known and
supported when parsing (rather than when attempting to unzip). It
removes a bunch of extension parsing from the code too, in favor of
doing it once upfront.
Closes https://github.com/astral-sh/uv/issues/5858.
## Summary
If we just created an entrypoint script, we can of course set the
permissions (we just created it). However, if we're copying from the
cache, we might _not_ own the file. In that case, if we need to change
the permissions (we shouldn't, since the script is likely already
executable -- we set the permissions when we unzip, but I guess they
could _not_ be properly set in the zip itself), we have to copy it.
Closes https://github.com/astral-sh/uv/issues/5581.
Zanie Blue
Zsolt Dollenstein
William Woodruff
Charlie Marsh
konsti
adamnemecek
Aria Desires
samypr100
Amos Wenger
Aditya Pratap Singh
Michał Górny