Commit Graph

5204 Commits

Author SHA1 Message Date
Charlie Marsh
8ff1dfd9b5 Add test cases for hash enforcement from existing uv.lock (#10081) 2024-12-21 09:53:38 -05:00
Charlie Marsh
705b3da913 Preserve sort when deciding on requirement placement (#10078)
## Summary

We had the right logic for determining whether the list is already
sorted, but we forgot to apply the same logic when deciding where to
insert the requirement, which made the list _unsorted_ for future
operations.

Closes https://github.com/astral-sh/uv/issues/10076.
2024-12-21 09:43:43 -05:00
Charlie Marsh
19a6b5fe4b Include hashes for local source archives (#10080)
## Summary

Closes https://github.com/astral-sh/uv/issues/10077
2024-12-21 09:31:28 -05:00
Charlie Marsh
3da4fdeeb3 Use portable path in requirements wire (#10071) 2024-12-20 23:10:24 -05:00
Charlie Marsh
9128fe1866 Use portable path instead of string for subdirectory (#10069)
## Summary

A few places where there are extra conversions to and from string that
seem unnecessary; a few places where we're using `PathBuf` instead of
`PortablePathBuf`.
2024-12-21 00:28:36 +00:00
Charlie Marsh
ddc290feb4 Add support for subdirectories in direct URLs in uv.lock (#10068)
## Summary

There were some subtle bugs here and no test coverage.
2024-12-20 19:00:50 -05:00
Charlie Marsh
2c68dfd4a9 Backtrack to non-local versions when wheels are missing platform support (#10046)
## Summary

This is yet another variation on
https://github.com/astral-sh/uv/pull/9928, with a few minor changes:

1. It only applies to local versions (e.g., `2.5.1+cpu`).
2. It only _considers_ the non-local version as an alternative (e.g.,
`2.5.1`).
3. It only _considers_ the non-local alternative if it _does_ support
the unsupported platform.
4. Instead of failing, it falls back to using the local version.

So, this is far less strict, and is effectively designed to solve
PyTorch but nothing else. It's also not user-configurable, except by way
of using `environments` to exclude platforms.
2024-12-20 19:11:27 +00:00
Charlie Marsh
f3c5b63095 Avoid duplicating backslashes in sysconfig parser (#10063)
## Summary

We had a bug in our handling of escape sequences that caused us to
duplicate backslashes. If you installed repeatedly, we'd keep doubling
them, leading to an exponential blowup.

Closes #10060.
2024-12-20 13:52:42 -05:00
bw513
2ca5c2ba70 Add additional version filter to mirror script. (#10055)
## Summary

Adds regular expression based version filter to python mirror script.

## Test Plan

Manually using `uv run ./scripts/create-python-mirror.py --name cpython
--arch x86_64 --os linux --version "3.13.\d+$"`
2024-12-20 12:50:59 -06:00
konsti
cf14a62de7 Remove uv-pep440 cdylib (#10058) 2024-12-20 15:38:13 +00:00
bw513
6af14bddca Fix mirror script to handle newer metadata format. (#10050)
## Summary

The architecture details in `crates/uv-python/download-metadata.json` is
now a dictionary with family and variant data, whereas it used to be a
string. This patches the architecture filter in
`scripts/create-python-mirror.py` to support both scenarios.

## Test Plan

Tested manually using `uv run ./scripts/create-python-mirror.py --name
cpython --arch x86_64 --os linux --from-all-history`
2024-12-20 13:37:51 +00:00
konsti
ae659c8bfe Stable order for virtual packages (#10024)
uv gives priorities to packages by package name, not by virtual package
(`PubGrubPackage`). pubgrub otoh when prioritizing order the virtual
packages. When the order of virtual packages changes, uv changes its
resolutions and error messages. This means uv was depending on
implementation details of pubgrub's prioritization caching.

This broke with https://github.com/pubgrub-rs/pubgrub/pull/299, which
added a tiebreaker term that made pubgrub's sorting deterministic given
a deterministic ordering of allocating the packages (which happens the
first time pubgrub sees a package).

The new custom tiebreaker decreases the difference to upstream pubgrub.
2024-12-20 09:28:46 +00:00
konsti
ff860296c5 Use shutil.which for the build backend (#10028)
From PEP 517:

> All command-line scripts provided by the build-required packages must
be present in the build environment’s PATH. For example, if a project
declares a build-requirement on flit, then the following must work as a
mechanism for running the flit command-line tool:
>
> ```python
> import subprocess
> import shutil
> subprocess.check_call([shutil.which("flit"), ...])
> ```

Fixes #9991

---------

Co-authored-by: Charles Tapley Hoyt <cthoyt@gmail.com>
2024-12-20 10:15:24 +01:00
Zanie Blue
c4d0caaee5 Bump version to 0.5.11 (#10044) 0.5.11 2024-12-19 18:01:45 -06:00
github-actions[bot]
7cfa1fd204 Sync latest Python releases (#10045)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2024-12-19 17:58:25 -06:00
Zanie Blue
8908e26de7 Add uv python install --preview to the documentation (#10010) 2024-12-19 16:07:46 -06:00
Charlie Marsh
2b0f8ffdee Remove remaining references to deprecated first-match (#10038)
## Summary

Missed these in https://github.com/astral-sh/uv/pull/10036.
2024-12-19 14:37:27 -05:00
Charlie Marsh
4513ce0e2a Allow --script to be provided with uv run - (#10035)
## Summary

Closes #10021.
2024-12-19 17:52:57 +00:00
Charlie Marsh
5a3826d9ff Remove references to deprecated first-match (#10036)
## Summary

This is a backwards-compatible alias for `first-index`.

Closes https://github.com/astral-sh/uv/issues/10031.
2024-12-19 12:40:14 -05:00
Charlie Marsh
5c6e584987 Allow uv run arguments when reading from stdin (#10034)
## Summary

Closes https://github.com/astral-sh/uv/issues/10033.
2024-12-19 12:39:56 -05:00
Steven L.
bd03243dd5 Fix typo in Caching docs (#10032)
Fix typo in Caching docs

Co-authored-by: Steven Locorotondo <steven.locorotondo@justeattakeaway.com>
2024-12-19 12:15:32 -05:00
konsti
557e750199 Build backend: Preserve executable bit (#10027)
Fixes #9968
2024-12-19 17:54:44 +01:00
FishAlchemist
afdcea6540 Supplement missing separators for UV_INSTALL_DIR in Windows. (#9507)
## Summary
Ensure proper separation between instructions.
fix #9503 
## Test Plan
Both Windows Powershell (v5) and powershell 7 are available

![image](https://github.com/user-attachments/assets/9a73ea6f-8bee-49a1-b6d4-050317176cd6)



<!-- How was it tested? -->
2024-12-19 09:53:49 -06:00
konsti
ac348eecdf Batch prefetch per fork (#10029)
Previously, the batch prefetcher was part of the solver loop, used
across forks. This would lead to each preference in a fork being counted
as a tried version, so that after 5 forks with the identical version, we
would start batch prefetching. The reported numbers of tried versions
are also reported. By tracking the batch prefetcher on the fork the
numbers are corrected.

An alternative would be tracking the actually tried versions, but that
would mean more overhead in the top level solver loop when the current
heuristic works.

In `ecosystem/transformers`:

```
$ hyperfine --runs 10 --prepare "rm -f uv.lock" "../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z" "uv lock --exclude-newer 2024-08-08T00:00:00Z"
Benchmark 1: ../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z
  Time (mean ± σ):     386.2 ms ±   6.1 ms    [User: 396.0 ms, System: 144.5 ms]
  Range (min … max):   378.5 ms … 397.9 ms    10 runs

Benchmark 2: uv lock --exclude-newer 2024-08-08T00:00:00Z
  Time (mean ± σ):     422.0 ms ±   5.5 ms    [User: 459.6 ms, System: 190.3 ms]
  Range (min … max):   415.0 ms … 430.5 ms    10 runs

Summary
  ../../target/release/uv lock --exclude-newer 2024-08-08T00:00:00Z ran
    1.09 ± 0.02 times faster than uv lock --exclude-newer 2024-08-08T00:00:00Z
```
2024-12-19 15:47:01 +01:00
Jp
dd442450b0 Retry mechanisms on Windows for copy_atomic and write_atomic (#10026)
Hello! 🙂 


## Summary

After submitting retry mechanisms on scripts installation for windows:
#9543 , I noticed that some other functions were using the same
`persist` features of temporary files. This could lead to the same issue
spotted before (temporary lock by AV/EDR software). I validated that it
was possible.

So I updated them to go through the same function on Windows, which is
using the retry mechanisms if needed.
In order to do so, I add to add an async version of the
`persist_with_retry`.

There is a little trick to make the borrow-checker happy line 306,
curious of your opinion on it? This is just a pointer move so it should
not induce some performance regression if I'm not mistaking.

I also updated them to use `fs_err` on Unix for better error messages.

Also, one of the error messages I introduced was badly formatted, I
fixed it. 🙂

## Test Plan

The changes should be iso functional and covered with the existing
test-suite.
2024-12-19 08:43:57 -05:00
samypr100
e65a273f1b Sync env vars with uv-static crate 🧹 (#10016)
## Summary

Updates some env vars references to use `EnvVars` accordingly.
2024-12-18 22:10:27 -05:00
Zanie Blue
1d8dac17fd Fix uv python install --default note about multiple requests (#10011) 2024-12-18 22:12:54 +00:00
Charlie Marsh
4d3c1b325e Prefer higher Python lower-bounds when forking (#10007)
## Summary

With the advent of `--fork-strategy requires-python` (the default), we
actually _want_ to solve higher lower-bound forks before lower
lower-bound forks. The former ensures we get the most compatible
versions, while the latter ensures we get fewer overall versions. These
two strategies match up with `--fork-strategy`, but need to be respected
as such.

Closes https://github.com/astral-sh/uv/issues/9998.
2024-12-18 16:54:56 -05:00
konsti
8f88f98350 Update packse to 0.3.42 for backtracking test (#10009)
Add the missing test for #9843.
2024-12-18 19:53:34 +00:00
konsti
cb325e2e2f metadata_directory already contains dist-info directory (#10005)
From PEP 517:

```python
def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
    ...
```

> Must create a .dist-info directory containing wheel metadata inside
the specified metadata_directory (i.e., creates a directory like
{metadata_directory}/{package}-{version}.dist-info/).

```python
def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
    ...
```

> If the build frontend has previously called
prepare_metadata_for_build_wheel and depends on the wheel resulting from
this call to have metadata matching this earlier call, then it should
provide the path to the created .dist-info directory as the
metadata_directory argument.

Notice that the `metadata_directory` is different for the both hooks:
For `prepare_metadata_for_build_wheel` is doesn't contain the
`.dist-info` directory as final segment, for `build_wheel` it does.

Previously, the code assumed that both directories didn't contain the
`.dist-info` for both cases.

Checked with:

```
maturin build
uv init test-uv-build-backend --build-backend uv
cd test-uv-build-backend
uv build --sdist --preview
cd ..
UV_PREVIEW=1 pip install test-uv-build-backend/dist/test_uv_build_backend-0.1.0.tar.gz --no-index --find-links target/wheels/ -v --no-cache-dir
```

Fixes #9969
2024-12-18 20:14:08 +01:00
Michał Górny
9305badcaf Fix show_settings tests not to be affected by system configs (#9992)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Override XDG_CONFIG_DIRS in show_settings tests, in order to ensure that
they don't pick system configuration, and therefore fail due to value
mismatches. This specifically addresses test failures on Gentoo where a
default `/etc/xdg/uv/uv.toml` is installed, and users are free to modify
it.

Prior to #9914, we used to set `XDG_CONFIG_DIRS` locally before running
the test suite. However, since the test now wipes the environment, the
problem can no longer be resolved downstream.

## Test Plan

`cargo test` on a Gentoo system (with `/etc/xdg/uv/uv.toml` present).

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-12-18 11:02:15 -05:00
Charlie Marsh
dd760ee507 Normalize platform_system to sys_platform (#9949)
## Summary

A revival of an old idea (#9344) that I have slightly more confidence in
now. I abandoned this idea because (1) it couldn't capture that, e.g.,
`platform_system == 'Windows' and sys_platform == 'foo'` (or some other
unknown value) are disjoint, and (2) I thought that Android returned
`"android"` for one of `sys_platform` or `platform_system`, which
would've made this logic incorrect.

However, it looks like Android... doesn't do that? And the values here
are almost always in a small, known set. So in the end, the tradeoffs
here actually seem pretty good.

Vis-a-vis our current solution, this can (e.g.) _simplify out_
expressions like `sys_platform == 'win32' or platform_system ==
'Windows'`.
2024-12-18 10:29:34 -05:00
github-actions[bot]
eb6bf8b0ee Sync latest Python releases (#9986)
Automated update for Python releases.

Co-authored-by: zanieb <2586601+zanieb@users.noreply.github.com>
2024-12-18 09:19:02 -06:00
Ben Beasley
45b9aa8f41 Make the backoff crate dependency Windows-only (#10002)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Since the `backoff` dependency is only *used* on Windows in practice,
this PR would ensure that it is only *compiled* on Windows, too. This is
helpful because it appears to be unmaintained upstream,
https://github.com/astral-sh/uv/issues/10001, and it would be nice to be
able to [drop it from
Fedora](https://bugzilla.redhat.com/show_bug.cgi?id=2329729).
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
```
$ cargo run python install
$ cargo test
```
2024-12-18 10:04:54 -05:00
Charlie Marsh
294da52610 Add nuance to prefetch logging (#9984) 2024-12-17 22:52:45 +00:00
Zanie Blue
37b11ddb22 Bump version to 0.5.10 (#9983) 0.5.10 2024-12-17 14:53:19 -06:00
Zanie Blue
026a40d2a4 Improve styling of uv remove dependency hints (#9960)
Instead of using a warning, which is pretty aggressive feeling, use a
hint.
2024-12-17 14:23:29 -06:00
Zanie Blue
d70160a57b Update references to python-build-standalone to reflect the transferred project (#9977) 2024-12-17 20:19:58 +00:00
samypr100
e730ef19f1 Patch additional sysconfig values such as clang at install time (#9916)
## Summary

Minor follow up to https://github.com/astral-sh/uv/pull/9905 to patch
`clang` with `cc`.

Implements the replacements used in
[sysconfigpatcher](https://github.com/bluss/sysconfigpatcher/blob/main/src/sysconfigpatcher.py#L54),
namely

```python
DEFAULT_VARIABLE_UPDATES = {
    "CC": WordReplace("clang", "cc"),
    "CXX": WordReplace("clang++", "c++"),
    "BLDSHARED": WordReplace("clang", "cc"),
    "LDSHARED": WordReplace("clang", "cc"),
    "LDCXXSHARED": WordReplace("clang++", "c++"),
    "LINKCC": WordReplace("clang", "cc"),
    "AR": "ar",
}
```

## Test Plan

Added an additional test. Tested local python installs.

Related traces
```
TRACE Updated `AR` from `/tools/clang-linux64/bin/llvm-ar` to `ar`
TRACE Updated `CC` from `clang -pthread` to `cc -pthread`
TRACE Updated `CXX` from `clang++ -pthread` to `c++ -pthread`
TRACE Updated `BLDSHARED` from `clang -pthread -shared -L/tools/deps/lib` to `cc -pthread -shared -L/tools/deps/lib`
TRACE Updated `LDSHARED` from `clang -pthread -shared -L/tools/deps/lib` to `cc -pthread -shared -L/tools/deps/lib`
TRACE Updated `LDCXXSHARED` from `clang++ -pthread -shared` to `c++ -pthread -shared`
TRACE Updated `LINKCC` from `clang -pthread` to `cc -pthread
```

## Pending Discussion Items

https://github.com/astral-sh/uv/pull/9905#issuecomment-2543879587
2024-12-17 15:09:45 -05:00
Zanie Blue
6dfe1774e8 Use the build options value to improve hints for no wheel / source distribution errors (#9950)
Extends the hints from https://github.com/astral-sh/uv/pull/9948 with
`BuildOptions` context so we can explain a bit more.
2024-12-17 14:08:22 -06:00
Zanie Blue
5e7f80ea46 Disable Depot runner for macOS (#9976)
Use GitHub instead; Depot do not provide the necessary concurrency to
use this here and in `python-build-standalone`
2024-12-17 11:20:47 -06:00
Zanie Blue
515d72c6f9 Improve display of ranges when pre-releases are not allowed (#9944)
Closes https://github.com/astral-sh/uv/issues/9891

There are two changes here

1. We now exclude pre-releases (if they are not allowed) from the
available versions set when simplifying ranges, this means the
simplified range reflects the _allowed_ available versions — which is
what we want. We no longer segment ranges into arbitrary looking
segments..
2. We improve on #9885, expanding the scope to avoid regressions where
we would now otherwise enumerate a bunch of versions

---------

Co-authored-by: konsti <konstin@mailbox.org>
2024-12-17 17:05:15 +00:00
Zanie Blue
3fae5332c1 Fix Cargo.lock by updating thiserror (#9975) 2024-12-17 10:53:44 -06:00
konsti
ebc6d20d9d Better build error messages (#9660)
Build failures are one of the most common user facing failures that
aren't "obivous" errors (such as typos) or resolver errors. Currently,
they show more technical details than being focussed on this being an
error in a subprocess that is either on the side of the package or -
more likely - in the build environment, e.g. the user needs to install a
dev package or their python version is incompatible.

The new error message clearly delineates the part that's important (this
is a build backend problem) from the internals (we called this hook) and
is consistent about which part of the dist building stage failed. We
have to calibrate the exact wording of the error message some more. Most
of the implementation is working around the orphan rule, (this)error
rules and trait rules, so it came out more of a refactoring than
intended.

Example:


![image](https://github.com/user-attachments/assets/2bc12992-db79-4362-a444-fd0d94594b77)
2024-12-17 09:44:32 -06:00
konsti
b7df5dbaf3 Avoid liblzma-dev system dep in uv-dev and uv-bench (#9933)
Enable `lzma-sys/static` through the performance feature not only in uv,
but in uv-dev and uv-bench too, to avoid the system dependency on
`liblzma-dev`.

Ref #9880
2024-12-17 16:12:33 +01:00
Zanie Blue
052c1a6fd1 Collapse redundant Python version incompatibilities in resolver error message (#9957)
Part of https://github.com/astral-sh/uv/issues/9886

Technically could affect other redundant clauses, but that does not
appear to be the case in practice.
2024-12-17 14:27:07 +00:00
konsti
654ff8015a Build backend: Fix pre-PEP 639 license files (#9965)
We were not copying the license file from a pre-PEP 639 declaration to
the source distribution.

Fixes #9947
2024-12-17 14:19:59 +00:00
Zanie Blue
a78e7468a7 Omit trailing zeros on Python requirements inferred from versions (#9952)
In a message like

```
❯ echo "numpy>2" | uv pip compile -p 3.8 -
  × No solution found when resolving dependencies:
  ╰─▶ Because the requested Python version (>=3.8.0) does not satisfy Python>=3.10 and the requested 
  Python version (>=3.8.0) does not satisfy Python>=3.9,<3.10, we can conclude that Python>=3.9 is incompatible.
      And because numpy>=2.0.1,<=2.0.2 depends on Python>=3.9 and only the following versions of numpy are available:
          numpy<=2.0.2
```

I'm surprised that `-p 3.8` leads to expressions like `>=3.8.0` (I
understand it, of course, but it's not intuitive) and then all the
_other_ Python versions in the message omit the trailing zero. This
updates the `PythonRequirement` parsing to drop the trailing zeros. It's
easier to do there because the version is not yet abstracted.
2024-12-17 08:18:27 -06:00
Zanie Blue
2288905d46 Improve error messages for uv remove (#9959)
Closes https://github.com/astral-sh/uv/issues/9958
2024-12-17 08:16:01 -06:00
Zanie Blue
686f383fa4 Improve phrasing for single term incompatibilities (#9953)
It makes more sense to say "cannot be used" rather than "is
incompatible" when the term is a single package
2024-12-17 08:14:33 -06:00