Commit Graph

855 Commits

Author SHA1 Message Date
konsti 8ce0736f9e
All (virtual) packages must have a priority (#10853)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-23 16:09:47 +00:00
Charlie Marsh b2dac9979d
Use Hashbrown's raw entry API to reduce hashes and clone in priority (#10881)
## Summary

I'm open to not merging this -- I was kind of just interested in what
the API looked like. But the idea is: we can avoid hashing values twice
and unnecessarily cloning within the priority map by using the raw entry
API.
2025-01-23 09:34:37 -05:00
Charlie Marsh c3a511716a
Remove dependencies clone in resolver (#10880) 2025-01-23 10:59:24 +01:00
Andrew Gallant a8d23da59c uv-resolver: error during installation for conflicting extras
This collects ALL activated extras while traversing the lock file to
produce a `Resolution` for installation. If any two extras are activated
that are conflicting, then an error is produced.

We add a couple of tests to demonstrate the behavior. One case is
desirable (where we conditionally depend on `package[extra]`) and the
other case is undesirable (where we create an uninstallable lock file).

Fixes #9942, Fixes #10590
2025-01-22 18:52:05 -05:00
Andrew Gallant 1676e63603 uv-resolver: fix propagation of extras
This will make `package[extra]` work even when `extra` is declared as a
conflicting extra.

Note that this isn't relevant for dependency groups since AFAIK those
can actually only be enabled on the CLI. There is no `package:group`
dependency syntax.
2025-01-22 18:52:05 -05:00
Andrew Gallant ae9c5c849d uv-resolver: remove the conservative checking of unconditional extras
This removes the error that was causing folks problems.

This does result in some snapshot updates that are arguably wrong, or at
least sub-optimal. However, it's actually intended. Because the approach
we're going to take is going to permit the creation of uninstallable
lock files as a side effect. In the future, we will modify this test to
check that, while `uv lock` succeeds, `uv sync` will always fail.
2025-01-22 18:52:05 -05:00
Charlie Marsh 088c194159
Remove clones from satisfies variants (#10876) 2025-01-22 16:22:07 -05:00
Charlie Marsh f5447ce965
Invalidate lockfile when static versions change (#10858)
## Summary

We should only be ignoring changes in `version` for dynamic projects;
for static projects, it should still be enforced. We should also be
invalidating the lockfile if a project goes from static to dynamic or
vice versa.

Closes #10852.
2025-01-22 17:58:15 +00:00
Charlie Marsh 434706389b
Error when workspace contains conflicting Python requirements (#10841)
## Summary

If members define disjoint Python requirements, we should error. Right
now, it seems that it maps to unbounded and leads to weird behavior.

Closes https://github.com/astral-sh/uv/issues/10835.
2025-01-22 17:22:52 +00:00
Charlie Marsh f0dc1f4dd1
Respect visitation order for proxy packages (#10833)
## Summary

This PR reverts https://github.com/astral-sh/uv/pull/10441 and applies a
different fix for https://github.com/astral-sh/uv/issues/10425.

In #10441, I changed prioritization to visit proxies eagerly. I think
this is actually wrong, since it means we prioritize proxy packages
above _everything_ else. And while a proxy only depends on itself, it
does mean we're selecting a _version_ for the proxy package earlier than
anything else. So, if you look at #10828, we end up choosing a version
for `async-timeout` before we choose a version for `langchain`, despite
the latter being a first-party dependency. (`async-timeout` has a marker
on it, so it has a proxy package, so we solve for it first.)

To fix #10425, we instead need to make sure we visit proxies in the
order we see them. I think the virtual tiebreaker for proxies is
reversed? We want to visit the package we see first, first.

So, in short: this reverts #10441, then corrects the ordering for
visiting proxies.

Closes https://github.com/astral-sh/uv/issues/10828.
2025-01-22 17:12:47 +00:00
Charlie Marsh e02f061ea9
Sort extras and groups when comparing lockfile requirements (#10856)
## Summary

The linked issue actually isn't a bug on main anymore, but it does
require us to take the "slow" path, since setuptools seems to reorder
the extras. This PR adds another normalization step which lets us take
the fast path: https://github.com/astral-sh/uv/issues/10855.
2025-01-22 12:06:05 -05:00
Charlie Marsh 62f8a483ef
Remove extra collects when flattening requirements (#10837) 2025-01-22 10:14:07 -05:00
Charlie Marsh a62b891e6a
Add fast-path for recursive extras in dynamic validation (#10823)
## Summary

Closes https://github.com/astral-sh/uv/issues/10816.
2025-01-21 16:01:20 -05:00
Charlie Marsh 9dd1217e1d
Remove clones from `PubGrubDependency::from_requirement` (#10821) 2025-01-21 15:32:34 -05:00
Charlie Marsh 78639187e8
Remove the `FullCommit` variant from `GitReference` (#10803)
## Summary

It's not quite correct for this to be on `GitReference`. It's not a
variant that can be created by users, or by us.
2025-01-21 19:26:58 +00:00
Andrew Gallant 9552c0a8db
uv-resolver: include conflict markers in fork markers (#10818)
When support for conflicting extras/groups was initially added, I
stopped short of including the conflict markers in uv's "fork markers"
in the lock file. That is, the fork markers are markers that indicate
the different splits uv took during resolution, which we record, I
believe, to avoid spurious updates to the lock file as a result of
using them as preferences.

One interesting result of omitting the conflict markers from the fork
markers is that sometimes this would result in duplicate markers. In
response, I wrote a function that stripped off the conflict markers and
deduplicated the remainder. My thinking at the time was that it wasn't
clear whether we needed to keep conflict markers around.

It looks like #10783 demonstrates a case where we do, seemingly, need
them. Namely, it's a case where after stripping conflict markers, you
don't end up with duplicate markers, but you do end up with overlapping
markers. Overlapping fork markers are bad juju for the same reason that
overlapping resolver forks are bad juju: you can end up with multiple
versions of the same package in the same environment.

I don't know how to fix overlapping markers without just including the
conflict markers. So that's what this PR does. Because of this, there
will be some churn in lock files, but this only applies to projects that
define conflicting extras.

This PR includes a regression test from #10783. I also manually tried
the original reproduction in #10772 (where adding `numpy<2` caused `uv
sync` to fail), and things worked.

Fixes #10772, Fixes #10783
2025-01-21 14:04:31 -05:00
Charlie Marsh 5c7fba86e1
Deduplicate `GitSha` and `GitOid` types (#10802)
## Summary

I think this split is leftover from using `libgit2`. I kept `Oid` since
that seems to be the official terminology.
2025-01-21 09:15:11 -05:00
Charlie Marsh 4354c0c9b7
Add fallback to build backend when `Requires-Dist` mismatches (#10797)
## Summary

This is a smaller alternative to #10794. If the `Requires-Dist` that we
extract statically doesn't match the lockfile metadata, we now go back
to the distribution database to double-check. Checking the
`Requires-Dist` is itself very cheap, so in the worst case, we're just
paying the same cost as prior to this optimization.

Closes https://github.com/astral-sh/uv/issues/10776.
2025-01-21 00:45:56 +00:00
Charlie Marsh 45911693c4
Remove short commit variant (#10795)
## Summary

This is never constructed.
2025-01-20 18:35:47 -05:00
Charlie Marsh 07e1e85c5d
Avoid deserialization error for paths above the root (#10789)
## Summary

Closes https://github.com/astral-sh/uv/issues/10777

## Test Plan

Copied over this lockfile:

```toml
version = 1
requires-python = ">=3.12"
resolution-markers = [
    "sys_platform == 'win32'",
    "sys_platform != 'win32'",
]

[[package]]
name = "pyasn1"
version = "0.6.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "01f1a642459aae6ee7b159a6c4c018/pyasn1-0.6.1.tar.gz", hash = "sha256:6f580d2bdd84365380830acf45550f2511469f673cb4a5ae3857a3170128b034", size = 145322 }
wheels = [
    { url = "d6a797abb18c925cab503dd37f8214/pyasn1-0.6.1-py3-none-any.whl", hash = "sha256:0d632f46f2ba09143da3a8afe9e33fb6f92fa2320ab7e886e2d0f7672af84629", size = 83135 },
]

[[package]]
name = "pyasn1-modules"
version = "0.4.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "pyasn1" },
]
sdist = { url = "6afbf0d507a72057e9c23797a737c9/pyasn1_modules-0.4.1.tar.gz", hash = "sha256:c28e2dbf9c06ad61c71a075c7e0f9fd0f1b0bb2d2ad4377f240d33ac2ab60a7c", size = 310028 }
wheels = [
    { url = "bc88a6711982eaa35a0a47c8032bdc/pyasn1_modules-0.4.1-py3-none-any.whl", hash = "sha256:49bfa96b45a292b711e986f222502c1c9a5e1f4e568fc30e2574a6c7d07838fd", size = 181537 },
]

[[package]]
name = "python-ldap"
version = "3.4.4"
source = { registry = "https://pypi.org/simple" }
resolution-markers = [
    "sys_platform != 'win32'",
]
dependencies = [
    { name = "pyasn1", marker = "sys_platform != 'win32'" },
    { name = "pyasn1-modules", marker = "sys_platform != 'win32'" },
]
sdist = { url = "1eeb4025dc4955b72db5ce7a4dbfbd/python-ldap-3.4.4.tar.gz", hash = "sha256:7edb0accec4e037797705f3a05cbf36a9fde50d08c8f67f2aef99a2628fab828", size = 377889 }

[[package]]
name = "python-ldap"
version = "3.4.4"
source = { path = "../../../uti/Python/python_ldap-3.4.4-cp312-cp312-win_amd64.whl" }
resolution-markers = [
    "sys_platform == 'win32'",
]
dependencies = [
    { name = "pyasn1", marker = "sys_platform == 'win32'" },
    { name = "pyasn1-modules", marker = "sys_platform == 'win32'" },
]
wheels = [
    { filename = "python_ldap-3.4.4-cp312-cp312-win_amd64.whl", hash = "sha256:94d2ca2b3ced81c9d89aa5c79d4965d03053e1ffdcfae73e9fac85d25b692e85" },
]

[package.metadata]
requires-dist = [
    { name = "pyasn1", specifier = ">=0.3.7" },
    { name = "pyasn1-modules", specifier = ">=0.1.5" },
]

[[package]]
name = "uv-test"
version = "1.0"
source = { virtual = "." }
dependencies = [
    { name = "python-ldap", version = "3.4.4", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'win32'" },
    { name = "python-ldap", version = "3.4.4", source = { path = "../../../uti/Python/python_ldap-3.4.4-cp312-cp312-win_amd64.whl" }, marker = "sys_platform == 'win32'" },
]

[package.metadata]
requires-dist = [
    { name = "python-ldap", marker = "sys_platform != 'win32'" },
    { name = "python-ldap", marker = "sys_platform == 'win32'", path = "../../../../../../../../../../../../uti/Python/python_ldap-3.4.4-cp312-cp312-win_amd64.whl" },
]
```

Verified that `cargo run sync --frozen` installs `python-ldap` from
PyPI, without erroring.
2025-01-20 16:36:18 -05:00
Charlie Marsh b2d06f01cc
Add tag incompatibility hints to sync failures (#10739)
## Summary

These are very similar to (and computed in the same way as) the hints we
should during a failed resolution, but for install-time.

Closes #10635.

## Test Plan

As an example, when installing PyTorch on macOS with Python 3.13 (wheels
exist for Linux):

```
error: Distribution `torch==2.5.1 @ registry+https://pypi.org/simple` can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're on macOS (`macosx_14_0_arm64`), but `torch` (v2.5.1) only has wheels for the following platform: `manylinux1_x86_64`
```
2025-01-20 12:46:46 -05:00
Charlie Marsh 8b6383ebe8
Avoid respecting preferences from other indexes (#10782)
## Summary

The fix I shipped in https://github.com/astral-sh/uv/pull/10690
regressed an important case. If we solve a PyPI branch before a PyTorch
branch, we'll end up respecting the preference, and choosing `2.2.2`
instead of `2.2.2+cpu`.

This PR goes back to ignoring preferences that don't map to the current
index. However, to solve https://github.com/astral-sh/uv/issues/10383,
we need to special-case `requirements.txt`, which can't provide explicit
indexes. So, if a preference comes from `requirements.txt`, we still
respect it.

Closes https://github.com/astral-sh/uv/issues/10772.
2025-01-20 12:23:00 -05:00
Charlie Marsh c306e46e1d
Remove trailing commas before brackets (#10740) 2025-01-18 19:56:46 +00:00
Charlie Marsh ae366ccd0e
Remove unnecessary clone on package name (#10741) 2025-01-18 19:45:19 +00:00
Charlie Marsh 35aec8863e
Use colors for lock errors (#10736)
## Summary

These now better match the errors we show when failing to resolve.
2025-01-18 18:50:20 +00:00
Charlie Marsh 3fe4e7168b
Use short-form for platform tag checks (#10737) 2025-01-18 18:22:09 +00:00
Charlie Marsh 1bfa7230fc
Correct documentation for wheel tag hints (#10735)
These were copy-pasted by accident.
2025-01-18 18:13:21 +00:00
Charlie Marsh ddff891397
Use iterators rather than eagerly collecting tags (#10734) 2025-01-18 13:11:48 -05:00
Charlie Marsh e02a7bb75d
Sort preferences by environment, then index (#10700)
## Summary

This has a few effects:

1. We only call `preferences` once, which should be more efficient.
2. We collect `preferences` into a vector when there are multiple. Less
efficient, but pretty rare?
3. We now correctly prefer preferences from the same index.
2025-01-17 17:27:44 +00:00
Charlie Marsh 8111650100
Avoid returning `Some` for narrowing no-ops (#10705)
The net effect here is (I think) just that we end up logging more
"narrowing" outcomes than we have in practice.
2025-01-17 16:41:17 +00:00
Charlie Marsh bc8002e26e
Avoid narrowing `requires-python` marker with disjunctions (#10704)
## Summary

A bug in `requires_python` (which infers the Python requirement from a
marker) was leading us to break an invariant around the relationship
between the marker environment and the Python requirement. This, in
turn, was leading us to drop parts of the environment space when
solving.

Specifically, in the linked example, we generated a fork for
`python_full_version < '3.10' or platform_python_implementation !=
'CPython'`, which was later split into `python_full_version == '3.8.*'`
and `python_full_version == '3.9.*'`, losing the
`platform_python_implementation != 'CPython'` portion.

Closes https://github.com/astral-sh/uv/issues/10669.
2025-01-17 16:25:32 +00:00
Charlie Marsh dce7b9da13
Store unsupported tags in wheel filename (#10665)
## Summary

We can retain the small-size advantage of our new tags by moving the
"unknown tag" case into `WheelTagLarge`. This ensures that we can still
represent unknown tags, but avoid paying the cost for them.
2025-01-17 04:41:53 +00:00
Charlie Marsh 80bdb3a997
Avoid building dynamic versions when validating lockfile (#10703)
## Summary

Closes #10689.
2025-01-17 04:27:46 +00:00
Charlie Marsh 45455b33c0
Respect preferences for explicit index dependencies from `requirements.txt` (#10690)
## Summary

Closes: https://github.com/astral-sh/uv/issues/10383.
2025-01-16 18:10:00 +00:00
Mathieu Kniewallner b46c6db317
docs: fix a few typos (#10675)
## Summary

Fixing a few typos found in the documentation and in comments.
2025-01-16 09:53:59 -05:00
Charlie Marsh ee6ba41d46
Add support for `pypy_73`-style tags (#10660)
## Summary

I'm inferring that these are like... the older tag format? See, e.g.:

```
soxbindings-0.0.1-pp27-pypy_73-macosx_10_9_x86_64.whl
soxbindings-0.0.1-pp27-pypy_73-manylinux2010_x86_64.whl
soxbindings-0.0.1-pp36-pypy36_pp73-macosx_10_9_x86_64.whl
soxbindings-0.0.1-pp36-pypy36_pp73-manylinux2010_x86_64.whl
```
2025-01-15 23:27:17 +00:00
Charlie Marsh ed34d37e3c
Avoid failing when deserializing unknown tags (#10655)
## Summary

Closes
https://github.com/astral-sh/uv/issues/10654#issuecomment-2594022975.
2025-01-15 18:03:28 -05:00
Charlie Marsh 04fc36f066
Show target Python version in error messages (#10582)
## Summary

See: https://github.com/astral-sh/uv/pull/10527#discussion_r1913593405

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-01-15 20:08:39 +00:00
Charlie Marsh 0617fd5da6
Omit dynamic versions from the lockfile (#10622)
## Summary

This PR modifies the lockfile to omit versions for source trees that use
`dynamic` versioning, thereby enabling projects to use dynamic
versioning with `uv.lock`.

Prior to this change, dynamic versioning was largely incompatible with
locking, especially for popular tools like `setuptools_scm` -- in that
case, every commit bumps the version, so every commit invalidates the
committed lockfile.

Closes https://github.com/astral-sh/uv/issues/7533.
2025-01-15 11:54:32 -05:00
Charlie Marsh 82c08b83d9
Make `version` an optional field on installable distribution type (#10623)
## Summary

I previously made this required, but we now need to be able to create
these from a lockfile that _omits_ versions for dynamic source trees.
They should still be present in most cases, but it's best-effort.
2025-01-15 11:31:39 -05:00
Charlie Marsh a49c3f1262
Show resolver hints for packages with markers (#10607)
## Summary

Closes https://github.com/astral-sh/uv/issues/10606.
2025-01-14 13:17:26 -05:00
Charlie Marsh e1e9b0447c
Reduce distribution size to 200 bytes (#10601)
## Summary

The last of these optimizations. The limit is now `RegistryBuiltDist`
which includes `sdist: Option<RegistrySourceDist>`.
2025-01-14 11:53:46 -05:00
Charlie Marsh d8b5e7e7c0
Use `ArcStr` in verbatim URL (#10600)
## Summary

No need to use `String` here.
2025-01-14 16:12:14 +00:00
Charlie Marsh 24a5920739
Reduce `WheelFilename` to 48 bytes (#10583)
## Summary

Based on some advice from @konstin.
2025-01-14 14:49:17 +00:00
Charlie Marsh e3f6b9c5f3
Filter wheels from lockfile based on architecture (#10584)
## Summary

After we resolve, we filter out any wheels that aren't applicable for
the target platforms. So, e.g., we remove macOS wheels if we find that
the user only asked to solve for Windows.

This PR extends the same logic to architectures, so that we filter out
ARM-only wheels when the user is only solving for x86, etc.

Closes #10571.
2025-01-14 14:39:21 +00:00
Charlie Marsh 3fd090b373
Correct Pyston tag format (#10580)
## Summary

Empirically, it looks like the format here is slightly different than
what we had in the code? Our integration test caught it.
2025-01-14 02:31:11 +00:00
Charlie Marsh 5c91217488
Use structured wheel tags everywhere (#10542)
## Summary

This PR extends the thinking in #10525 to platform tags, and then uses
the structured tag enums everywhere, rather than passing around strings.
I think this is a big improvement! It means we're no longer doing ad hoc
tag parsing all over the place.
2025-01-14 01:39:39 +00:00
Charlie Marsh 2ffa31946d
Show expected and available ABI tags in resolver errors (#10527)
## Summary

The idea here is to show both (1) an example of a compatible tag and (2)
the tags that were available, whenever we fail to resolve due to an
abscence of matching wheels.

Closes https://github.com/astral-sh/uv/issues/2777.
2025-01-14 01:03:11 +00:00
samypr100 4d3809cc6b
Upgrade Rust toolchain to 1.84.0 (#10533)
## Summary
Upgrade the rust toolchain to 1.84.0. This PR does not bump the MSRV.
2025-01-11 22:19:33 -05:00
Charlie Marsh 5788cd2b18
Fix typo in `version_map.rs` (#10528) 2025-01-11 23:39:46 +00:00