Commit Graph

817 Commits

Author SHA1 Message Date
Charlie Marsh 0617fd5da6
Omit dynamic versions from the lockfile (#10622)
## Summary

This PR modifies the lockfile to omit versions for source trees that use
`dynamic` versioning, thereby enabling projects to use dynamic
versioning with `uv.lock`.

Prior to this change, dynamic versioning was largely incompatible with
locking, especially for popular tools like `setuptools_scm` -- in that
case, every commit bumps the version, so every commit invalidates the
committed lockfile.

Closes https://github.com/astral-sh/uv/issues/7533.
2025-01-15 11:54:32 -05:00
Charlie Marsh 82c08b83d9
Make `version` an optional field on installable distribution type (#10623)
## Summary

I previously made this required, but we now need to be able to create
these from a lockfile that _omits_ versions for dynamic source trees.
They should still be present in most cases, but it's best-effort.
2025-01-15 11:31:39 -05:00
Charlie Marsh a49c3f1262
Show resolver hints for packages with markers (#10607)
## Summary

Closes https://github.com/astral-sh/uv/issues/10606.
2025-01-14 13:17:26 -05:00
Charlie Marsh e1e9b0447c
Reduce distribution size to 200 bytes (#10601)
## Summary

The last of these optimizations. The limit is now `RegistryBuiltDist`
which includes `sdist: Option<RegistrySourceDist>`.
2025-01-14 11:53:46 -05:00
Charlie Marsh d8b5e7e7c0
Use `ArcStr` in verbatim URL (#10600)
## Summary

No need to use `String` here.
2025-01-14 16:12:14 +00:00
Charlie Marsh 24a5920739
Reduce `WheelFilename` to 48 bytes (#10583)
## Summary

Based on some advice from @konstin.
2025-01-14 14:49:17 +00:00
Charlie Marsh e3f6b9c5f3
Filter wheels from lockfile based on architecture (#10584)
## Summary

After we resolve, we filter out any wheels that aren't applicable for
the target platforms. So, e.g., we remove macOS wheels if we find that
the user only asked to solve for Windows.

This PR extends the same logic to architectures, so that we filter out
ARM-only wheels when the user is only solving for x86, etc.

Closes #10571.
2025-01-14 14:39:21 +00:00
Charlie Marsh 3fd090b373
Correct Pyston tag format (#10580)
## Summary

Empirically, it looks like the format here is slightly different than
what we had in the code? Our integration test caught it.
2025-01-14 02:31:11 +00:00
Charlie Marsh 5c91217488
Use structured wheel tags everywhere (#10542)
## Summary

This PR extends the thinking in #10525 to platform tags, and then uses
the structured tag enums everywhere, rather than passing around strings.
I think this is a big improvement! It means we're no longer doing ad hoc
tag parsing all over the place.
2025-01-14 01:39:39 +00:00
Charlie Marsh 2ffa31946d
Show expected and available ABI tags in resolver errors (#10527)
## Summary

The idea here is to show both (1) an example of a compatible tag and (2)
the tags that were available, whenever we fail to resolve due to an
abscence of matching wheels.

Closes https://github.com/astral-sh/uv/issues/2777.
2025-01-14 01:03:11 +00:00
samypr100 4d3809cc6b
Upgrade Rust toolchain to 1.84.0 (#10533)
## Summary
Upgrade the rust toolchain to 1.84.0. This PR does not bump the MSRV.
2025-01-11 22:19:33 -05:00
Charlie Marsh 5788cd2b18
Fix typo in `version_map.rs` (#10528) 2025-01-11 23:39:46 +00:00
Charlie Marsh 54b3a438d0
Avoid forking for identical markers (#10490)
## Summary

If you have a dependency with a marker, and you add a constraint, it
causes us to _always_ fork, because we represent the constraint as a
second dependency with the marker repeated (and, therefore, we have two
requirements of the same name, both with markers). I don't think we
should fork here -- and in the end it's leading to this undesirable
resolution: #10481.

I tried to change constraints such that we just _reuse_ and augment the
initial requirement, but that has a fairly negative effect on error
messages: #10489. So this fix seems a bit better to me.

Closes https://github.com/astral-sh/uv/issues/10481.
2025-01-10 22:30:04 -05:00
Charlie Marsh 8420195aa7
Use `ArcStr` for marker values (#10453)
N.B. After fixing #10430, `ArcStr` became the fastest implementation
(and the gains were significantly reduced, down to 1-2%). See:
https://github.com/astral-sh/uv/pull/10453#issuecomment-2583344414.

## Summary

I tried out a variety of small string crates, but `Arc<str>`
outperformed them, giving a ~10% speed-up:

```console
❯ hyperfine "../arcstr lock" "../flexstr lock" "uv lock" "../arc lock" "../compact_str lock" --prepare "rm -f uv.lock" --min-runs 50 --warmup 20
Benchmark 1: ../arcstr lock
  Time (mean ± σ):     304.6 ms ±   2.3 ms    [User: 302.9 ms, System: 117.8 ms]
  Range (min … max):   299.0 ms … 311.3 ms    50 runs

Benchmark 2: ../flexstr lock
  Time (mean ± σ):     319.2 ms ±   1.7 ms    [User: 317.7 ms, System: 118.2 ms]
  Range (min … max):   316.8 ms … 323.3 ms    50 runs

Benchmark 3: uv lock
  Time (mean ± σ):     330.6 ms ±   1.5 ms    [User: 328.1 ms, System: 139.3 ms]
  Range (min … max):   326.6 ms … 334.2 ms    50 runs

Benchmark 4: ../arc lock
  Time (mean ± σ):     303.0 ms ±   1.2 ms    [User: 301.6 ms, System: 118.4 ms]
  Range (min … max):   300.3 ms … 305.3 ms    50 runs

Benchmark 5: ../compact_str lock
  Time (mean ± σ):     320.4 ms ±   2.0 ms    [User: 318.7 ms, System: 120.8 ms]
  Range (min … max):   317.3 ms … 326.7 ms    50 runs

Summary
  ../arc lock ran
    1.01 ± 0.01 times faster than ../arcstr lock
    1.05 ± 0.01 times faster than ../flexstr lock
    1.06 ± 0.01 times faster than ../compact_str lock
    1.09 ± 0.01 times faster than uv lock
```
2025-01-10 15:15:12 -05:00
Charlie Marsh b3d7beb1a0
Use `arcstr` for package, extra, and group names (#10475)
## Summary

This appears to be a consistent 1% performance improvement and should
also reduce memory quite a bit. We've also decided to use these for
markers, so it's nice to use the same optimization here.

```
❯ hyperfine "./uv pip compile --universal scripts/requirements/airflow.in" "./arcstr pip compile --universal scripts/requirements/airflow.in" --min-runs 50 --warmup 20
Benchmark 1: ./uv pip compile --universal scripts/requirements/airflow.in
  Time (mean ± σ):     136.3 ms ±   4.0 ms    [User: 139.1 ms, System: 241.9 ms]
  Range (min … max):   131.5 ms … 149.5 ms    50 runs

Benchmark 2: ./arcstr pip compile --universal scripts/requirements/airflow.in
  Time (mean ± σ):     134.9 ms ±   3.2 ms    [User: 137.6 ms, System: 239.0 ms]
  Range (min … max):   130.1 ms … 151.8 ms    50 runs

Summary
  ./arcstr pip compile --universal scripts/requirements/airflow.in ran
    1.01 ± 0.04 times faster than ./uv pip compile --universal scripts/requirements/airflow.in
```
2025-01-10 19:46:36 +00:00
Andrew Gallant 503f9a97af
uv-resolver: pre-compute PEP 508 markers from universal markers (#10472)
It turns out that we use `UniversalMarker::pep508` quite a bit. To the
point that it makes sense to pre-compute it when constructing a
`UniversalMarker`.

This still isn't necessarily the fastest thing we can do, but this
results in a major speed-up and `without_extras` no longer shows up for
me in a profile.

Motivating benchmarks. First, from #10430:

```
$ hyperfine 'rm -f uv.lock && uv lock' 'rm -f uv.lock && uv-ag-optimize-without-extras lock'
Benchmark 1: rm -f uv.lock && uv lock
  Time (mean ± σ):     408.3 ms ± 276.6 ms    [User: 333.6 ms, System: 111.1 ms]
  Range (min … max):   316.9 ms … 1195.3 ms    10 runs

  Warning: The first benchmarking run for this command was significantly slower than the rest (1.195 s). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Benchmark 2: rm -f uv.lock && uv-ag-optimize-without-extras lock
  Time (mean ± σ):     209.4 ms ±   2.2 ms    [User: 209.8 ms, System: 103.8 ms]
  Range (min … max):   206.1 ms … 213.4 ms    14 runs

Summary
  rm -f uv.lock && uv-ag-optimize-without-extras lock ran
    1.95 ± 1.32 times faster than rm -f uv.lock && uv lock
```

And now from #10438:

```
$ hyperfine 'uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null' 'uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null'
Benchmark 1: uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
  Time (mean ± σ):     12.718 s ±  0.052 s    [User: 12.818 s, System: 0.140 s]
  Range (min … max):   12.650 s … 12.815 s    10 runs

Benchmark 2: uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
  Time (mean ± σ):     419.5 ms ±   6.7 ms    [User: 434.7 ms, System: 100.6 ms]
  Range (min … max):   412.7 ms … 434.3 ms    10 runs

Summary
  uv-ag-optimize-without-extras pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null ran
   30.32 ± 0.50 times faster than uv pip compile requirements.in -c constraints.txt --universal --no-progress --python-version 3.8 --offline > /dev/null
```

Fixes #10430, Fixes #10438
2025-01-10 11:23:19 -05:00
Charlie Marsh bee2baa64e
Misc. changes based on ABI explorations (#10458) 2025-01-10 08:10:39 -05:00
Charlie Marsh 7bf514d886
Remove `get_with_version` methods (#10456)
## Summary

I think these are vestigial.
2025-01-10 03:32:30 +00:00
Charlie Marsh 7096e83812
Respect sentinels in prioritization (#10443)
## Summary

If a user provides a constraint like `flask==3.0.0`, that gets expanded
to `[3.0.0, 3.0.0+[max])`. So it's not a _singleton_, but it should be
treated as such for the purposes of prioritization, since in practice it
will almost always map to a single version.
2025-01-09 21:19:49 +00:00
Charlie Marsh 56d39d21c2
Visit proxy packages eagerly (#10441)
## Summary

The issue here is that we add `urllib3{python_full_version >= '3.8'}` as
a dependency, then `requests{python_full_version >= '3.8'}`, which adds
`urllib3`, but at that point, we haven't expanded
`urllib3{python_full_version >= '3.8'}`, so we "lose" the singleton
constraint. The solution is to ensure that we visit proxies eagerly, so
that we accumulate constraints as early as possible.

Closes
https://github.com/astral-sh/uv/issues/10425#issuecomment-2580324578.
2025-01-09 20:37:28 +00:00
Charlie Marsh f65fcf23b3
Remove duplicated comment (#10416) 2025-01-08 23:06:09 +00:00
Charlie Marsh 4c161d284b
Respect `requires-python` when installing tools (#10401)
## Summary

This PR revives https://github.com/astral-sh/uv/pull/7827 to improve
tool resolutions such that, if the resolution fails, and the selected
interpreter doesn't match the required Python version from the solve, we
attempt to re-solve with a newly-discovered interpreter that _does_
match the required Python version.

For now, we attempt to choose a Python interpreter that's greater than
the inferred `requires-python`, but compatible with the same Python
minor. This helps avoid successive failures for cases like Posting,
where choosing Python 3.13 fails because it has a dependency that lacks
source distributions and doesn't publish any Python 3.13 wheels. We
should further improve the strategy to solve _that_ case too, but this
is at least the more conservative option...

In short, if you do `uv tool instal posting`, and we find Python 3.8 on
your machine, we'll detect that `requires-python: >=3.11`, then search
for the latest Python 3.11 interpreter and re-resolve.

Closes https://github.com/astral-sh/uv/issues/6381.
Closes https://github.com/astral-sh/uv/issues/10282.

## Test Plan

The following should succeed:

```
cargo run python uninstall --all
cargo run python install 3.8
cargo run tool install posting
```

In the logs, we see:

```
...
DEBUG No compatible version found for: posting
DEBUG Refining interpreter with: Python >=3.11, <3.12
DEBUG Searching for Python >=3.11, <3.12 in managed installations or search path
DEBUG Searching for managed installations at `/Users/crmarsh/.local/share/uv/python`
DEBUG Skipping incompatible managed installation `cpython-3.8.20-macos-aarch64-none`
DEBUG Found `cpython-3.13.1-macos-aarch64-none` at `/opt/homebrew/bin/python3` (search path)
DEBUG Skipping interpreter at `/opt/homebrew/opt/python@3.13/bin/python3.13` from search path: does not satisfy request `>=3.11, <3.12`
DEBUG Found `cpython-3.11.7-macos-aarch64-none` at `/opt/homebrew/bin/python3.11` (search path)
DEBUG Re-resolving with Python 3.11.7
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.7
DEBUG Solving with target Python version: >=3.11.7
DEBUG Adding direct dependency: posting*
DEBUG Searching for a compatible version of posting (*)
...
```
2025-01-08 12:38:17 -05:00
konsti c5583b326f
Shrink `Dist` from 352 to 288 bytes (#10389)
Found this when looking at #10385. Since we're constructing a lot of
`Dist`s, we should keep it small.
2025-01-08 09:33:19 -05:00
konsti 68adadf806
Improve file pinning comments (#10387) 2025-01-08 11:42:25 +00:00
konsti 14a9008e54
Avoid overcounting versions in batch prefetcher (#10350) 2025-01-07 14:12:18 +00:00
konsti 3cbf8ab7b1
Optimize `requirements_for_extra` (#10348) 2025-01-07 14:12:12 +00:00
konsti c6ac121ed0
Refactor batch prefetch (#10349) 2025-01-07 14:58:36 +01:00
konsti 3dc481b063
Speed up file pins (#10346)
Ref https://github.com/astral-sh/uv/issues/10344

Avoid the nested hashmap, file pinning is called in the version
selection hot loop.

```
$ hyperfine --warmup 1 --prepare "uv venv -p 3.12" "./uv-2 pip compile scripts/requirements/airflow.in" "./uv-1 pip compile scripts/requirements/airflow.in"
    Finished `profiling` profile [optimized + debuginfo] target(s) in 0.12s
Benchmark 1: ./uv-2 pip compile scripts/requirements/airflow.in
  Time (mean ± σ):     420.1 ms ±   4.7 ms    [User: 585.4 ms, System: 195.1 ms]
  Range (min … max):   413.1 ms … 429.2 ms    10 runs

Benchmark 2: ./uv-1 pip compile scripts/requirements/airflow.in
  Time (mean ± σ):     473.0 ms ±   4.9 ms    [User: 654.4 ms, System: 209.4 ms]
  Range (min … max):   468.0 ms … 481.1 ms    10 runs

Summary
  ./uv-2 pip compile scripts/requirements/airflow.in ran
    1.13 ± 0.02 times faster than ./uv-1 pip compile scripts/requirements/airflow.in
```
2025-01-07 08:51:08 -05:00
konsti d8fcf2a58f
Simplify `requirements_for_extra` (#10347)
Ref https://github.com/astral-sh/uv/issues/10344

Not a performance optimization, but the function had become too large.
No logic changes, just code moving around. Looks slightly better when
ignoring whitespace changes.

It's still too complex but i haven't found an apt simplification.
2025-01-07 08:50:13 -05:00
konsti 180a138161
Deactivate tracing for choose version (#10351) 2025-01-07 09:55:06 +00:00
Charlie Marsh 0d57d298e6
Avoid downgrading packages when `--upgrade` is provided (#10097)
## Summary

When `--upgrade` is provided, we should retain already-installed
packages _if_ they're newer than whatever is available from the
registry.

Closes https://github.com/astral-sh/uv/issues/10089.
2025-01-06 17:41:43 -05:00
Charlie Marsh 66a603b6c4
Use dynamic dispatch to simplify reporters (#10086)
## Summary

Sort of undecided on this. These are already stored as `dyn Reporter` in
each struct, so we're already using dynamic dispatch in that sense. But
all the methods take `impl Reporter`. This is sometimes nice (the
callsites are simpler?), but it also means that in practice, you often
_can't_ pass `None` to these methods that accept `Option<impl
Reporter>`, because Rust can't infer the generic type.

Anyway, this adds more consistency and simplifies the setup by using
`Arc<dyn Reporter>` everywhere.
2025-01-06 12:04:00 -05:00
Charlie Marsh 833519d5d8
Extract supported architectures from wheel tags (#10179)
## Summary

This PR extends #10046 to also handle architectures, which allows us to
correctly include `2.5.1` on the `cu124` index for ARM Linux.

Closes https://github.com/astral-sh/uv/issues/9655.
2025-01-04 02:42:17 +00:00
Charlie Marsh 2eba52f674
Avoid forking on version in non-universal resolutions (#10274)
## Summary

Closes https://github.com/astral-sh/uv/issues/10275.
2025-01-02 11:40:48 -05:00
Charlie Marsh ed4e4ff4b5
Use `BTreeMap::range` to avoid iterating over unnecessary versions (#10266)
## Summary

Closes https://github.com/astral-sh/uv/issues/6174.
2025-01-02 01:21:36 +00:00
Charlie Marsh 7bbec6b2e4
Parse URLs lazily in resolver (#10259)
## Summary

The idea here is to avoid parsing all registry URLs upfront, and instead
parse them when we need them.

Closes #6133.
2025-01-01 12:35:30 -05:00
Charlie Marsh cf88828e55
Avoid stripping query parameters from URLs (#10253)
## Summary

Closes https://github.com/astral-sh/uv/issues/10251.
2024-12-31 12:59:26 -05:00
Charlie Marsh dcd96a83aa
Respect static metadata for already-installed distributions (#10242)
## Summary

Closes
https://github.com/astral-sh/uv/issues/10239#issuecomment-2565663046
2024-12-30 12:47:06 -05:00
Charlie Marsh b316d780bf
Consider workspace dependencies to be 'direct' (#10197)
## Summary

Closes https://github.com/astral-sh/uv/issues/7863.
2024-12-28 20:32:45 -05:00
Charlie Marsh d3f06de4f5
Fix incorrect mismatched constraints reference (#10184) 2024-12-26 23:25:38 +00:00
Charlie Marsh 79dce7391e
Avoid need for universal markers in `requirements.txt` export (#10171) 2024-12-26 15:32:32 +00:00
Charlie Marsh bec8468183
Remove unnecessary prefixes (#10158) 2024-12-25 14:18:01 -05:00
Charlie Marsh 3cb723220e
Remove `anyhow::Result` for lock serialization (#10151) 2024-12-25 01:24:26 +00:00
Charlie Marsh facd21a679
Replace `Option<BTreeSet>` with `BTreeSet` for members (#10150) 2024-12-25 00:37:46 +00:00
Charlie Marsh f872b56124
Add a lock target abstraction (#10129)
## Summary

This PR introduces a `LockTarget`, which is peer to `InstallTarget` and
enables us to capture the common functionality necessary to support
locking.

For now, to minimize changes, only the `Workspace` target is
implemented. In a future PR, I'll add a `Script` target for both locking
and installing.
2024-12-24 19:26:31 -05:00
Charlie Marsh 7c47a457d9
Move installable targets out of `uv-resolver` crate (#10126)
## Summary

The proximate motivation is that I want to add new variant for scripts,
but `uv-resolver` can't depend on `uv-scripts` without creating a
circular dependency. However, I think this _does_ just make more sense
-- the resolver crate shouldn't be coupled to the various kinds of
workspaces, and these details are mostly encoded in `projects/lock.rs`
and similar files.
2024-12-24 19:01:50 -05:00
Charlie Marsh 6745a8b00a
Show non-project dependencies in `uv tree` (#10149)
## Summary

Closes #10147.
2024-12-24 23:34:58 +00:00
Charlie Marsh e09b1080f4
Store dependency groups separate from dependencies in lockfile (#10148)
## Summary

This is necessary for some future improvements to non-`[project]`
workspaces and PEP 723 scripts. It's not "breaking", but it will
invalidate lockfiles for non-`[project]` workspaces. I think that's
okay, since we consider those legacy right now, and they're really rare.
2024-12-24 17:32:29 -05:00
Charlie Marsh 3435777e87
Support `uv export` for non-project workspaces (#10144) 2024-12-24 15:23:28 +00:00
Charlie Marsh 33cb3497aa
Strip fragment when storing direct URL (#10093)
## Summary

Closes
https://github.com/astral-sh/uv/issues/10088#issuecomment-2558280467.
2024-12-22 09:07:04 -05:00