Commit Graph

713 Commits

Author SHA1 Message Date
Andrew Gallant 19a044d4db uv-pypi-types: rename "conflicting group" types to more generic "conflicts"
Since this is intended to support _both_ groups and extras, it doesn't
make sense to just name it for groups. And since there isn't really a
word that encapsulates both "extra" and "group," we just fall back to
the super general "conflicts."

We'll rename the variables and other things in the next commit.
2024-11-14 08:02:46 -05:00
Charlie Marsh 9339e55a11
Add `version` to `ResolvedDist` (#9102)
## Summary

I need this for the derivation chain work
(https://github.com/astral-sh/uv/issues/8962), but it just seems
generally useful. You can't always get a version from a `Dist` (it could
be URL-based!), but when we create a `ResolvedDist`, we _do_ know the
version (and not just the URL). This PR preserves it.
2024-11-13 19:06:16 -05:00
Charlie Marsh 35549de62d
Defer reporting of build failures in resolver (#9098)
## Summary

In https://github.com/astral-sh/uv/issues/9078, resolution fails because
we fail to build `jsmin`. However... if you look at what's actually
happening, `jsmin` fails to build during _prefetching_. And we never
actually attempt to access its metadata later on.

This PR modifies the metadata result handling such that we don't raise
these errors until the resolver actually asks for the metadata, so
https://github.com/astral-sh/uv/issues/9078 now succeeds.

I actually had to make this change anyway in pursuing
https://github.com/astral-sh/uv/issues/8962, so I've decided to carve it
out here.

Closes https://github.com/astral-sh/uv/issues/9078.
2024-11-13 15:49:08 -05:00
Andrew Gallant 15ef807c80
add support for specifying conflicting extras (#8976)
This PR adds support for conflicting extras. For example, consider
some optional dependencies like this:

```toml
[project.optional-dependencies]
project1 = ["numpy==1.26.3"]
project2 = ["numpy==1.26.4"]
```

These dependency specifications are not compatible with one another.
And if you ask uv to lock these, you'll get an unresolvable error.

With this PR, you can now add this to your `pyproject.toml` to get
around this:

```toml
[tool.uv]
conflicting-groups = [
    [
      { package = "project", extra = "project1" },
      { package = "project", extra = "project2" },
    ],
]
```

This will make the universal resolver create additional forks
internally that keep the dependencies from the `project1` and
`project2` extras separate. And we make all of this work by reporting
an error at **install** time if one tries to install with two or more
extras that have been declared as conflicting. (If we didn't do this,
it would be possible to try and install two different versions of the
same package into the same environment.)

This PR does *not* add support for conflicting **groups**, but it is
intended to add support in a follow-up PR.

Closes #6981

Fixes #8024

Ref #6729, Ref #6830

This should also hopefully unblock
https://github.com/dagster-io/dagster/pull/23814, but in my testing, I
did run into other problems (specifically, with `pywin`). But it does
resolve the problem with incompatible dependencies in two different
extras once you declare `test-airflow-1` and `test-airflow-2` as
conflicting for `dagster-airflow`.

NOTE: This PR doesn't make `conflicting-groups` public yet. And in a
follow-up PR, I plan to switch the name to `conflicts` instead of
`conflicting-groups`, since it will be able to accept conflicting extras
_and_ conflicting groups.
2024-11-13 09:52:28 -05:00
Charlie Marsh b3c660c58a
Rename `Fetch` to `Download` in build errors (#9039)
## Summary

We're inconsistent with these -- sometimes it's `Error::Fetch` and
sometimes it's `Error::Download`. The message says download, so let's
just use that?
2024-11-12 02:30:20 +00:00
Charlie Marsh 59f3ec2b6c
Remove duplicate `Lock::to_resolution` (#9029)
## Summary

This got moved to `InstallTarget`! Must've been an oversight not to
delete. I verified that no code was changed here since the date that we
moved it to `InstallTarget`.
2024-11-11 20:30:00 +00:00
Charlie Marsh dd4b27bfbd
Allow `--no-build` to validate lock (#9024)
## Summary

Just as we don't enforce tag compliance, we shouldn't enforce
`--no-build` when validating the lockfile. If we end up building from
source, the distribution database will correctly error.

Closes https://github.com/astral-sh/uv/issues/9016.
2024-11-11 14:02:37 -05:00
Charlie Marsh ac9ce853cf
Remove some unused public methods (#8993)
## Summary

Notice these in an unrelated refactor.
2024-11-10 22:55:22 +00:00
Charlie Marsh 744a9091a2
Allow default indexes to be marked as explicit (#8990)
## Summary

Closes https://github.com/astral-sh/uv/issues/8985.
2024-11-10 18:05:39 +00:00
Charlie Marsh 8a3e5d43e6
Fix references to `--resolution-strategy` in error message output (#8971)
Closes https://github.com/astral-sh/uv/issues/8967.
2024-11-09 13:54:49 +00:00
Charlie Marsh 363c589f76
Prefer compatible to incompatible distributions when packages exist on multiple indexes (#8961)
## Summary

At time of writing, `markupsafe==3.0.2` exists on the PyTorch index, but
there's
only a single wheel:


`MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`

Meanwhile, there are a large number of wheels on PyPI for the same
version. If the
user is on Python 3.12, and we return the incompatible PyTorch wheel
without
considering the PyPI wheels, PubGrub will mark 3.0.2 as an incompatible
version,
even though there are compatible wheels on PyPI.

Closes https://github.com/astral-sh/uv/issues/8922.
2024-11-08 19:48:48 -05:00
Charlie Marsh 75c26229a2
Support multiple extras in universal pip compile output (#8960)
## Summary

We were making some incorrect assumptions in the extra-merging code for
universal `pip compile`. This PR corrects those assumptions and adds a
bunch of additional tests.

Closes https://github.com/astral-sh/uv/issues/8915.
2024-11-08 18:22:29 -05:00
Charlie Marsh 90653e1f5e Remove all special-casing for local version identifiers (#8818)
After https://github.com/astral-sh/uv/pull/8797, we have spec-compliant
handling for local version identifiers and can completely remove all the
special-casing around it.
2024-11-07 14:29:54 -06:00
Eric Mark Martin c49c7bdf97 Implement PEP 440-compliant local version semantics (#8797)
Implement a full working version of local version semantics. The (AFAIA)
major move towards this was implemented in #2430. This added support
such that the version specifier `torch==2.1.0+cpu` would install
`torch@2.1.0+cpu` and consider `torch@2.1.0+cpu` a valid way to satisfy
the requirement `torch==2.1.0` in further dependency resolution.

In this feature, we more fully support local version semantics. Namely,
we now allow `torch==2.1.0` to install `torch@2.1.0+cpu` regardless of
whether `torch@2.1.0` (no local tag) actually exists.

We do this by adding an internal-only `Max` value to local versions that
compare greater to all other local versions. Then we can translate
`torch==2.1.0` into bounds: greater than 2.1.0 with no local tag and
less than 2.1.0 with the `Max` local tag.

Depends on https://github.com/astral-sh/packse/pull/227.
2024-11-07 14:29:54 -06:00
Zanie Blue 4874b32d85 do not imply pre-release when `!=` operator is used (#7974)
closes #6640

Could you suggest how I should test it?

(already tested locally)

---------

Co-authored-by: konstin <konstin@mailbox.org>
Co-authored-by: Charles Tapley Hoyt <cthoyt@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-07 14:29:54 -06:00
Charlie Marsh 29e1b15473
Add `uv tree --outdated` (#8893)
## Summary

Similar to `pip list --outdated`, but for `uv tree`.

## Test Plan

Looks like:

```
foo v0.1.0
└── flask v2.0.0 (latest: v3.0.3)
    ├── click v8.1.7
    ├── itsdangerous v2.2.0
    ├── jinja2 v3.1.4
    │   └── markupsafe v3.0.2
    └── werkzeug v3.1.2
        └── markupsafe v3.0.2
```

With `(latest: v3.0.3)` in bold cyan.
2024-11-07 14:10:46 -06:00
Charlie Marsh 987d778631
Respect dynamic version updates in `uv lock` (#8867)
## Summary

Closes https://github.com/astral-sh/uv/issues/8866.
2024-11-06 11:40:23 -05:00
Charlie Marsh 26e3511ebd
Respect fork markers in `--resolution-mode=lowest-direct` (#8839)
## Summary

Previously, given:

```toml
dependencies = [
    "pycountry >= 22.1.10",
    "setuptools >= 50.0.0; python_version>='3.12'"
]
```

We'd solve for the lowest version of setuptools (with _no_ lower-bound
constraint) in the `python_version < '3.12'` complement.

Closes https://github.com/astral-sh/uv/issues/8819.
2024-11-05 21:09:38 +00:00
Andrew Gallant acaed763b7 uv: use ResolverEnvironment instead of ResolverMarkers
This updates the surrounding code to use the new ResolverEnvironment
type. In some cases, this simplifies caller code by removing case
analysis. There *shouldn't* be any behavior changes here. Some test
snapshots were updated to account for some minor tweaks to error
messages.

I didn't split this up into separate commits because it would have been
too difficult/costly.
2024-11-04 11:09:06 -05:00
Andrew Gallant 44c9ef6aea uv-resolver: add new ResolverEnvironment type
This type is intended to replace `ResolverMarkers`. The main difference
between them is that this type encapsulates more decision making by
un-exporting the different cases. So instead of callers needing to do
explicit case analysis depending on the type of resolver environment,
callers instead use methods that know how to do the right thing. In the
next commit, there are at least a few cases where this greatly
simplifies case analysis on the caller side.

The motivation for this type is to centralize decision making about
forking. In particular, we want to expand forking to include conflicting
groups instead of just `MarkerTree`. So to a certain extent, the
refactor here is about removing bare use of `MarkerTree` in favor of a
more purpose built type that encapsulates the forking logic.

The encapsulation is not quite perfect here. I expect to improve on it a
bit once we add support for conflicting groups.

This is split off from the subsequent commit (that makes use of
`ResolverEnvironment`) so that it's a bit easier to review the addition
in isolation.
2024-11-04 11:09:06 -05:00
Tim de Jager ef8724c979
Add `From` for `FlatDistributions` struct (#8800)
Co-authored-by: konsti <konstin@mailbox.org>
2024-11-04 08:41:39 +00:00
Charlie Marsh bf79d985ee
Allow incompatible `requires-python` for source distributions with static metadata (#8768)
## Summary

At present, when we have a Python requirement and we see a wheel, we
verify that the Python requirement is compatible with the wheel. For
source distributions, though, we verify that both the Python requirement
_and_ the currently-installed version are compatible, because we assume
that we'll need to build the source distribution in order to get
metadata. However, we can often extract source distribution metadata
_without_ building (e.g., if there's a `pyproject.toml` with no dynamic
keys).

This PR thus modifies the source distribution handling to defer that
incompatibility ("We couldn't get metadata for this project, because it
has no static metadata and requires a higher Python version to run /
build") until we actually try to build the package. As a result, you can
now resolve source distribution-only packages using Python versions
below their `requires-python`, as long as they include static metadata.

Closes https://github.com/astral-sh/uv/issues/8767.
2024-11-03 19:03:55 +00:00
Charlie Marsh d3e50a2376
Enable `--all-packages --frozen` in `uv sync` and `uv export` (#8760)
## Summary

This PR improves the interaction of `--frozen` such that we reduce the
dependency on the `pyproject.toml` and increase the dependency on the
`uv.lock`. Specifically, we now read the list of workspace members from
the `uv.lock` rather than the `pyproject.toml`, which means we don't
need to discover the member `pyproject.toml` files in order to perform a
`uv sync --frozen --all-packages`.
2024-11-02 02:48:55 +00:00
Charlie Marsh b36ae6d5ae
Add support for `uv export --all-packages` (#8742)
## Summary

Same as the other PRs, but for `uv export`.
2024-11-02 02:25:29 +00:00
Charlie Marsh 3c9dd97fe9
Add support for `uv sync --all-packages` (#8739)
## Summary

This PR enables `uv sync --all-packages` to sync all packages in a
workspace. It removes a common use-case for the legacy non-`[project]`
packages that we're trying to move away from.

Closes https://github.com/astral-sh/uv/issues/8724.
2024-11-02 01:55:08 +00:00
Andrew Gallant 58a9811881
fix bug where `python_version < '0'` could appear in a final resolution (#8759)
This PR fixes a bug where it was possible for dependencies to be
included in a final resolution with markers that always evaluate to
false. Specifically, `python_version < '0'`.

While we do filter based on Python markers during forking, it turns out
that the markers for each fork are "combined" *after* this filtering
step. But the process of combination can result in a more specific
marker that is always false for the configured Python requirement. This
could result in dependencies with markers that are always false (like
`python_version < '0'`) appearing in the resolution.

The first commit in this PR adds a regression test (with an undesirable
result), and the second commit fixes the regression and updates the
test.

Fixes #8676
2024-11-01 20:11:40 +00:00
Charlie Marsh 24ad43e79c
Avoid showing dependency group annotations on workspace members in tree (#8730)
## Summary

By default, `uv tree` shows the full workspace, not _just_ the root. If
the root depended on a workspace member as a dev dependency, then we'd
still show it as `(group: dev)` in `uv tree` even if you passed
`--no-dev`, because we weren't filtering the edges in the right place.

This is still somewhat confusing, because if `root` depends on workspace
member `child` as a dev dependency, `uv tree --no-dev` still shows both.

Closes https://github.com/astral-sh/uv/issues/8719.
2024-10-31 16:09:36 -04:00
Charlie Marsh bed47d512a
Use portable paths for subdirectories in lock URLs (#8707)
## Summary

We're writing different URLs on Windows vs. Linux because the
subdirectory path is being serialized with a back vs. forward slash.
2024-10-30 15:37:40 -04:00
konsti c1a0fb35e8
Simplify pep440 -> version ranges conversion (#8683) 2024-10-30 13:10:48 +01:00
Charlie Marsh cd408cbc51
Clarify relationship between specifiers and `requires-python` range (#8688) 2024-10-29 22:30:01 +01:00
Charlie Marsh dc5e35ee5d
Fix outdated documentation on `Requires-Python` (#8679)
## Summary

Closes https://github.com/astral-sh/uv/issues/8675.
2024-10-29 20:18:43 +00:00
konsti e5b8cdba70
Merge uv-pubgrub into uv-pep440 (#8669) 2024-10-29 20:15:18 +01:00
Andrew Gallant 9fcc21236e
uv-resolver: simplify case analysis in ForkMap::get (#8672)
Previously, when doing a `uv pip` resolution, we would only return the
first entry in the map. But there should only ever be one entry, or else
we would have incompatible dependencies. So we can collapse the case
with the "one universal fork" case.

(I found this while doing some refactoring of how we handle forking, and
collapsing these cases simplifies some of that refactoring work.)
2024-10-29 13:38:21 -04:00
Andrew Gallant a56e521179
uv-resolver: use disjointness checks instead of marker equality (#8661)
When this code was written, we didn't have "proper" disjointness checks,
and so simple equality was used instead. Arguably disjointness checks
are
more correct, and this would also simplify some case analysis in an
ongoing
refactor.
2024-10-29 11:25:18 -04:00
Charlie Marsh bf14b6a282
Respect dependency group markers in `uv export` (#8659)
## Summary

Closes https://github.com/astral-sh/uv/issues/8658.
2024-10-29 13:41:27 +00:00
Charlie Marsh debe67ffdb
Use `dev-dependencies` and `requires-dev` for lockfile compatibility (#8599)
## Summary

Unfortunately, it looks like we lost
https://github.com/astral-sh/uv/pull/8501 somewhere in a bad rebase.
This PR re-adds the change, with compatibility for those lockfiles
created in v0.4.27. I'm not certain we should actually merge this. It
might be less painful and confusing to just bite the bullet on the
change.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-10-28 22:08:33 +00:00
Charlie Marsh bfa84cd1ab
Fork when minimum Python version increases (#8628)
## Summary

This is a re-implementation of
https://github.com/astral-sh/uv/pull/4712, though is now seemingly much
simpler. This issue keeps coming up, and users have a workaround with
`tool.uv.environments`, but it's really a bug in the resolver.

Closes https://github.com/astral-sh/uv/issues/4668.
2024-10-28 09:48:04 -04:00
Charlie Marsh 001d00ba26
Remove inverse-node tracking in `uv tree` (#8610)
## Summary

Just simplifying some of the representations.
2024-10-27 20:44:32 -04:00
Charlie Marsh f23d9c1a21
Support cyclic dependencies in `uv tree` (#8564)
## Summary

Closes https://github.com/astral-sh/uv/issues/8551.
2024-10-26 13:30:42 -04:00
Charlie Marsh 5ef0e6c838
Ignore empty groups when validating lock (#8598)
## Summary

It turns out we were omitting empty dependency groups from the lockfile
metadata, which was then causing us to reject locks when empty groups
were defined.

We now include them (that section of the lock is meant to be a true
representation of the metadata, and an empty-but-defined group is
different from an absent group), though we can ignore them for
validation, since it doesn't affect any behavior.

Closes https://github.com/astral-sh/uv/issues/8581.
2024-10-26 17:29:07 +00:00
Charlie Marsh dd0f696695
Allow `[dependency-groups]` in non-`[project]` projects (#8574)
## Summary

We already support `tool.uv.dev-dependencies` in the legacy
non-`[project]` projects. This adds equivalent support for
`[dependency-groups]`, e.g.:

```toml
[tool.uv.workspace]

[dependency-groups]
lint = ["ruff"]
```
2024-10-25 13:57:06 -05:00
Charlie Marsh 291c4c496d Add support for `default-groups` (#8471)
This PR adds support for `tool.uv.default-groups`, which defaults to
`["dev"]` for backwards-compatibility. These represent the groups we
sync by default.
2024-10-25 13:27:37 -05:00
Charlie Marsh 2e028cd3b6 Rewrite some references to "optional groups" (#8454)
## Summary

We generally want to avoid references to "optional groups" now that
dependency groups are a first-class, standardized concept.
2024-10-25 13:27:37 -05:00
Charlie Marsh 5ab860be20 Rename dev dependencies to dependency groups in lockfile (#8391)
This is backwards compatible (we respect `dev-dependencies` as an
alias).

Part of https://github.com/astral-sh/uv/pull/8272.
2024-10-25 13:27:37 -05:00
Zanie Blue 1bee82329c Add `--group`, `--only-group`, and `--only-dev` support to `uv tree` (#8338)
Part of #8090

Most of the heavy lifting is done in #8309

Includes `--only-dev` which appears to be missing as an oversight.
2024-10-25 13:27:37 -05:00
Zanie Blue d2e1f180ef Refactor development dependency configuration (#8309)
Part of #8090
Unblocks https://github.com/astral-sh/uv/pull/8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
2024-10-25 13:27:37 -05:00
Zanie Blue 4c0590ff6f Add `--group` support to `uv add` and `uv remove` (#8108)
Part of #8090

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
2024-10-25 13:27:37 -05:00
Charlie Marsh f4a3f97fe0
Fix dangling non-platform dependencies in `uv tree` (#8532)
## Summary

We were including dependencies that were only included by a dependency
that isn't relevant on the current platform (i.e., we were enforcing the
"current environment" at one level, but not transitively).

Closes https://github.com/astral-sh/uv/issues/8516.
2024-10-24 16:39:00 +00:00
Charlie Marsh 2651aee33f
Enforce lockfile schema versions (#8509)
## Summary

Historically, we haven't enforced schema versions. This PR adds a
versioning policy such that, if a uv version writes schema v2, then...

- It will always reject lockfiles with schema v3 or later.
- It _may_ reject lockfiles with schema v1, but can also choose to read
them, if possible.

(For example, the change we proposed to rename `dev-dependencies` to
`dependency-groups` would've been backwards-compatible: newer versions
of uv could still read lockfiles that used the `dev-dependencies` field
name, but older versions should reject lockfiles that use the
`dependency-groups` field name.)

Closes https://github.com/astral-sh/uv/issues/8465.
2024-10-24 12:23:56 -04:00
Charlie Marsh e9c08b1639
Rewrite `uv tree` to allow arbitrary `--package` includes (#8507)
## Summary

Previously, `uv tree --package` had some strange behavior due to how we
were computing the root nodes. This PR refactors the entire
implementation to use `petgraph` so we can do proper operations on a
graph structure.

Closes https://github.com/astral-sh/uv/issues/8382.
2024-10-23 18:43:26 +00:00
Charlie Marsh cc734ea2b6
Allow dependency metadata entries for direct URL requirements (#7846)
## Summary

This is part of making
https://github.com/astral-sh/uv/issues/7299#issuecomment-2385286341
better. You can now use `tool.uv.dependency-metadata` for direct URL
requirements. Unfortunately, you _must_ include a version, since we need
one to perform resolution.
2024-10-22 22:01:23 -04:00
Charlie Marsh 3542251c76
Apply narrowing with upper bounds (#8403)
## Summary

If the user has an upper-bound in a `requires-python`, we don't
correctly narrow it during resolution. We should be narrowing based on
the intersection.

Closes #8297.
2024-10-21 01:03:50 +00:00
Charlie Marsh 7e2822d694
Avoid panic when Git dependencies are included in fork markers (#8388)
## Summary

Rather than relying on the distribution and package URL being the same
(which isn't true for Git dependencies), we can just use the
intersection of the markers directly.

Closes https://github.com/astral-sh/uv/issues/8381.
2024-10-20 18:42:21 +00:00
Zanie Blue fff2094a35
Use a dedicated message for incompatible Python versions in wheel ABI tags (#8363)
Part of https://github.com/astral-sh/uv/issues/2777

I noticed we're seeing "Python ABI" _a lot_ in error messages which I
did not expect. This improves a common case by being a little more
specific.
2024-10-20 12:14:11 -04:00
konsti 32bba9f33b
Don't prefetch unreachable packages (#8246) 2024-10-18 13:44:24 +02:00
Charlie Marsh c62f8d769c
Redact index credentials from lockfile sources (#8307)
## Summary

Closes https://github.com/astral-sh/uv/issues/8296.
2024-10-17 21:38:21 +00:00
Charlie Marsh 4ca158931a
Show hint in resolution failure on `Forbidden` (`403`) or `Unauthorized` (`401`) (#8264)
## Summary

Closes https://github.com/astral-sh/uv/issues/8167.
2024-10-16 17:34:29 +00:00
Charlie Marsh 5e05a62004
Respect index priority when storing credentials (#8256)
## Summary

Closes https://github.com/astral-sh/uv/issues/8248.
2024-10-16 15:52:26 +00:00
Ian Paul e71b1d0c42
Warn when patch isn't specified (#7959)
When patch version isn't specified and a matching version is referenced,
it will default patch to 0 which could be unclear/confusing. This PR
warns the user of that default.

<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

The first part of this issue
https://github.com/astral-sh/uv/issues/7426. Will tackle the second part
mentioned (`~=`) in a separate PR once I know this is the correct way to
warn users.

## Test Plan

<!-- How was it tested? -->

Unit tests were added

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-10-16 04:21:26 +00:00
Charlie Marsh 2153c6ac0d
Respect named `--index` and `--default-index` values in `tool.uv.sources` (#7910)
## Summary

If you pass a named index via the CLI, you can now reference it as a
named source. This required some surprisingly large refactors, since we
now need to be able to track whether a given index was provided on the
CLI vs. elsewhere (since, e.g., we don't want users to be able to
reference named indexes defined in global configuration).

Closes https://github.com/astral-sh/uv/issues/7899.
2024-10-15 23:56:24 +00:00
Charlie Marsh a034a8b83b
Remove the flat index types (#7759)
## Summary

I think these really don't pull their weight.
2024-10-15 23:30:37 +00:00
Charlie Marsh 9a76e47888
Allow multiple pinned indexes in `tool.uv.sources` (#7769)
## Summary

This PR lifts the restriction that a package must come from a single
index. For example, you can now do:

```toml
[project]
name = "project"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["jinja2"]

[tool.uv.sources]
jinja2 = [
    { index = "torch-cu118", marker = "sys_platform == 'darwin'"},
    { index = "torch-cu124", marker = "sys_platform != 'darwin'"},
]

[[tool.uv.index]]
name = "torch-cu118"
url = "https://download.pytorch.org/whl/cu118"

[[tool.uv.index]]
name = "torch-cu124"
url = "https://download.pytorch.org/whl/cu124"
```

The construction is very similar to the way we handle URLs today: you
can have multiple URLs for a given package, but they must appear in
disjoint forks. So most of the code is just adding that abstraction to
the resolver, following our handling of URLs.

Closes #7761.
2024-10-15 22:58:15 +00:00
Charlie Marsh 1925922770
Enable environment variable authentication for named indexes (#7741)
## Summary

This PR enables users to provide index credentials via named environment
variables.

For example, given an index named `internal` that requires a username
(`public`) and password
(`koala`), you can define the index (without credentials) in your
`pyproject.toml`:

```toml
[[tool.uv.index]]
name = "internal"
url = "https://pypi-proxy.corp.dev/simple"
```

Then set the `UV_INDEX_INTERNAL_USERNAME` and
`UV_INDEX_INTERNAL_PASSWORD`
environment variables, where `INTERNAL` is the uppercase version of the
index name:

```sh
export UV_INDEX_INTERNAL_USERNAME=public
export UV_INDEX_INTERNAL_PASSWORD=koala
```
2024-10-15 22:35:07 +00:00
Charlie Marsh 5b391770df
Add support for named and explicit indexes (#7481)
## Summary

This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.

Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```

You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.

### Index priority

Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.

Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.

You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.

### Index pinning

If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"

[tool.uv.sources]
torch = { index = "pytorch" }
```

If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true

[tool.uv.sources]
torch = { index = "pytorch" }
```

Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.

(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)

### Default index

By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```

### Name reuse

If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.

For example, given:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"

[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```

The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.

Closes #171.

## Future work

- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
  { index = "cpu", markers = "sys_platform == 'darwin'" },
  { index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
2024-10-15 18:24:23 -04:00
Charlie Marsh c683191408
Don't recommend `--prerelease=allow` for source dist builds (#8192)
## Summary

Closes https://github.com/astral-sh/uv/issues/3686.
2024-10-14 21:04:30 -04:00
samypr100 01c44af3c3
chore: unify all env vars used (#8151)
## Summary

This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.

I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.

This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.

I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.

I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.

Did I miss any? Are their initial docs cohesive?

Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.

## Test Plan

Existing tests
2024-10-14 16:48:13 -05:00
Charlie Marsh 346050bf99
Improve major-minor bounds on requires-python (#8145) 2024-10-12 15:48:03 +01:00
Charlie Marsh b12d5b619b
Use shared index when fetching metadata in lock satisfaction routine (#8147) 2024-10-12 15:46:55 +01:00
bluss e67d87301a
Implement `uv tree --no-dev` (#8109)
## Summary

Allow pruning dev-dependencies in uv tree.
This is not inherently in conflict with --invert, but this pruning is
not yet implemented there.
2024-10-12 13:10:56 +00:00
Charlie Marsh b91bd29970
Avoid excluding valid wheels for exact `requires-python` bounds (#8140)
## Summary

Closes https://github.com/astral-sh/uv/issues/8136.
2024-10-12 04:17:36 +00:00
Amos Wenger 715f28fd39
chore: Move all integration tests to a single binary (#8093)
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html

Before that, there were 91 separate integration tests binary.

(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
2024-10-11 16:41:35 +02:00
Charlie Marsh 2506c1c274
Capitalize error messages from lockfile (#8115)
## Summary

This is more consistent with how we format errors everywhere else.
2024-10-11 01:50:00 +02:00
Charlie Marsh 7bac708b97
Treat resolver failures as fatal in lockfile validation (#8083)
## Summary

In the routine we use to verify whether the lockfile is up-to-date, we
sometimes have to resolve package metadata. If that resolution step
fails, the resolver is left in a bad state, as various tasks are marked
as pending despite the error. Treating that as a recoverable failure
thus leads to a deadlock.

This PR modifies the errors to be treated as fatal.

I think a more holistic fix here would be to add some kind of guard to
ensure that any tasks that fail are no longer marked as pending (or
enforce this in the type system).

Closes https://github.com/astral-sh/uv/issues/8074.
2024-10-10 14:01:20 +00:00
Charlie Marsh 1c5309080b
Add gap-preserving range-to-PEP 440 routine (#8060)
## Summary

These are changes I apparently forgot to push as per
https://github.com/astral-sh/uv/pull/7897/files#r1794312988.
2024-10-09 22:48:53 +00:00
Charlie Marsh 77ea9d9626
Fix handling of != intersections in `requires-python` (#7897)
## Summary

The issue here is that, if you user has a `requires-python` like `>=
3.7, != 3.8.5`, this gets expanded to the following bounds:

- `[3.7, 3.8.5)`
- `(3.8.5, ...`

We then convert this to the specific `>= 3.7, < 3.8.5, > 3.8.5`. But the
commas in that expression are conjunctions... So it's impossible to
satisfy? No version is both `< 3.8.5` and `> 3.8.5`.

Instead, we now preserve the input `requires-python` and just
concatenate the terms, only using PubGrub to compute the _bounds_.

Closes https://github.com/astral-sh/uv/issues/7862.
2024-10-10 00:24:43 +02:00
Charlie Marsh 0eb4320394
Respect project upper bounds when filtering wheels on requires-python (#7904)
## Summary

If the user sets an upper-bound on their `requires-python`, we can omit
more wheels.
2024-10-04 11:35:50 +01:00
Charlie Marsh 312eeb8f57
Always ignore `cp2` wheels in resolution (#7902)
## Summary

Closes #7873.
2024-10-03 17:35:03 +00:00
konsti 41fdecf457
Allow `py3x-none` tags in newer than Python 3.x (#7867)
Unlike `cp36-...`, which requires exactly CPython 3.6, `py36-none` is
compatible with all versions starting at Python 3.6.

Note that `py3x-none` should not be used. Instead, use `py3-none` with
`requires-python`.

Fixes #7800
2024-10-03 18:02:14 +01:00
Charlie Marsh 8962bcb028
Simplify supported environments when comparing to lockfile (#7894)
## Summary

If a supported environment includes a Python marker, we don't simplify
it out, despite _storing_ the simplified markers. This PR modifies the
validation code to compare simplified to simplified markers.

Closes https://github.com/astral-sh/uv/issues/7876.
2024-10-03 14:15:07 +01:00
Jo c07cdc6161
Remove the first empty line for `uv tree --package foo` (#7885)
## Summary

When using `uv tree --package foo`, an extra empty line appears at the
beginning, which seems unnecessary since `uv tree` without the package
option doesn’t have this. It’s possible that the intention was to add
separation between packages, i.e. the correct implementation shoule be:

```rust
if !std::mem::take(&mut first) {
    lines.push(String::new());
}
```

Even if corrected, this extra spacing might be redundant as `uv tree`
doesn’t include these empty lines between packages by default.

```console
$ uv init project
$ cd project
$ uv init foo
$ uv tree
Using CPython 3.12.5
Resolved 2 packages in 1ms
foo v0.1.0
project v0.1.0

$ uv tree --package project
Using CPython 3.12.5
Resolved 2 packages in 1ms

project v0.1.0
```
2024-10-03 13:04:36 +01:00
Charlie Marsh 14507a1793
Add `uv-` prefix to all internal crates (#7853)
## Summary

Brings more consistency to the repo and ensures that all crates
automatically show up in `--verbose` logging.
2024-10-01 20:15:32 -04:00
Tim de Jager 50116efb10
expose `FlatDistributions` struct in public API (#7833)
Would it be okay to expose this struct? We currently use our own
ResolveProvider, and it would be nice to use the `FlatDistributions` for
easy `VersionMap` creation.

Thanks!
2024-10-01 07:46:32 -05:00
Charlie Marsh 1602b5c8d7
Remove unnecessary index location methods (#7826) 2024-10-01 04:44:53 +00:00
Jo da9e85cc6a
Fix `uv tree --invert` for platform dependencies (#7808)
## Summary

`click` has one dependency of `colorama` only on Windows, `uv tree
--invert` should not include `colorama` on non-Windows platforms, but
currently:

```console
$ uv init
$ uv add click
$ uv tree --invert --python-platform macos
colorama v0.4.6
```

it should:
```console
$ uv tree --invert --python-platform macos
click v8.1.7
    └── project v0.1.0
```
2024-09-30 12:45:24 -04:00
Charlie Marsh fe88c10813
Remove unused `thiserror` variants in resolver (#7717)
## Summary

While looking at something else, I noticed that these are not used.
2024-09-26 16:36:52 +00:00
Liam c8357b7bf2
Allow unused mut in graph resolution conflicts (#7701) 2024-09-26 09:17:43 +00:00
Topher Anderson 84e5f6e871
Regression fix: don't prefetch source dists with unbounded lower-bound ranges (#7683)
#7226 modified the check to skip prefetching of source dists without
proper minimum-version bounds, and wound up flipping the boolean
expression. This change flips the some/none expression so that the
intended skip happens as expected.

Fixes #7680.
2024-09-25 08:35:19 -04:00
konsti 5da73a24cb
Rename `MetadataResolver` to `ResolutionMetadata` (#7661) 2024-09-24 16:25:19 +00:00
konsti 484717d42f
Split metadata parsing into a module (#7656) 2024-09-24 17:16:21 +02:00
Andrew Gallant 83f1abdf57 uv-resolver: add error checking for conflicting distributions
This PR adds some additional sanity checking on resolution graphs to
ensure we can never install different versions of the same package into
the same environment.

I used code similar to this to provoke bugs in the resolver before the
release, but it never made it into `main`. Here, we add the error
checking to the creation of `ResolutionGraph`, since this is where it's
most convenient to access the "full" markers of each distribution.

We only report an error when `debug_assertions` are enabled to avoid
rendering `uv` *completely* unusuable if a bug were to occur in a
production binary. For example, maybe a conflict is detected in a marker
environment that isn't actually used. While not ideal, `uv` is still
usable for any other marker environment.

Closes #5598
2024-09-24 10:55:23 -04:00
Charlie Marsh b14696ca7c
Show a dedicated PubGrub hint for `--unsafe-best-match` (#7645)
## Summary

Closes https://github.com/astral-sh/uv/issues/5510.
2024-09-23 19:02:19 +00:00
Luca Bruno 2c6d353711
Provide resolution hints in case of possible local name conflicts (#7505)
This enhances the hints generator in the resolver with some heuristic to
detect and warn in case of failures due to version mismatches on a local
package. Those may be the symptom of name conflict/shadowing with a
transitive dependency.

Closes: https://github.com/astral-sh/uv/issues/7329

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-09-20 13:07:34 -05:00
Luca Bruno 248bef13bd
Compute resolver hints using the final reduced derivation tree (#7546)
This switches the no-solution formatter to use the final derivation tree
(simplified and reduced) when generating hints.
2024-09-19 16:06:55 +02:00
Andrew Gallant 1379b530f6 uv: migrate to rkyv 0.8
Recently, rkyv 0.8 was released. Its API is a fair bit simpler now for
higher level uses (like for us in `uv`) and results in us being able to
delete a fair bit of code. This also removes our last dependency on `syn
1.0`, and thus drops that dependency.

Performance (via testing on the `transformers` example) seems to remain
about the same, which is what was expected:

```
$ hyperfine -w5 -r100 'uv lock' 'uv-ag-rkyv-update lock'
Benchmark 1: uv lock
  Time (mean ± σ):      55.6 ms ±   6.4 ms    [User: 30.4 ms, System: 35.1 ms]
  Range (min … max):    43.0 ms …  73.1 ms    100 runs

Benchmark 2: uv-ag-rkyv-update lock
  Time (mean ± σ):      56.5 ms ±   7.2 ms    [User: 30.5 ms, System: 36.3 ms]
  Range (min … max):    39.1 ms …  71.5 ms    100 runs

Summary
  uv lock ran
    1.02 ± 0.18 times faster than uv-ag-rkyv-update lock
```

Closes #7415
2024-09-18 14:49:54 -04:00
Luca Bruno 67cfb4a9c0
Use a single buffer for hints on resolver errors (#7497)
This changes the structure of the hints generator in the resolver when
encountering solution errors, so that it re-uses a single output buffer
owned by the caller.
It avoids repeated allocations of a temporary buffer within each
recursive function call.
2024-09-18 16:16:22 +02:00
Charlie Marsh fda227616c
Allow users to provide pre-defined metadata for resolution (#7442)
## Summary

This PR enables users to provide pre-defined static metadata for
dependencies. It's intended for situations in which the user depends on
a package that does _not_ declare static metadata (e.g., a
`setup.py`-only sdist), and that is expensive to build or even cannot be
built on some architectures. For example, you might have a Linux-only
dependency that can't be built on ARM -- but we need to build that
package in order to generate the lockfile. By providing static metadata,
the user can instruct uv to avoid building that package at all.

For example, to override all `anyio` versions:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]

[[tool.uv.dependency-metadata]]
name = "anyio"
requires-dist = ["iniconfig"]
```

Or, to override a specific version:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]

[[tool.uv.dependency-metadata]]
name = "anyio"
version = "3.7.0"
requires-dist = ["iniconfig"]
```

The current implementation uses `Metadata23` directly, so we adhere to
the exact schema expected internally and defined by the standards. Any
entries are treated similarly to overrides, in that we won't even look
for `anyio@3.7.0` metadata in the above example. (In a way, this also
enables #4422, since you could remove a dependency for a specific
package, though it's probably too unwieldy to use in practice, since
you'd need to redefine the _rest_ of the metadata, and do that for every
package that requires the package you want to omit.)

This is under-documented, since I want to get feedback on the core ideas
and names involved.

Closes https://github.com/astral-sh/uv/issues/7393.
2024-09-18 03:18:05 +00:00
Charlie Marsh 778da3350a
Add `--no-editable` support to `uv sync` and `uv export` (#7371)
## Summary

Closes https://github.com/astral-sh/uv/issues/5792.
2024-09-17 14:50:36 +00:00
Charlie Marsh c87ce7aaf8
Run `cargo upgrade` (#7448)
Co-authored-by: konstin <konstin@mailbox.org>
2024-09-17 12:39:58 +02:00
Charlie Marsh 424ee439d6
Use consistent PyPI cache bucket (#7443)
## Summary

All the registry wheels were getting cached under
`index/b2a7eb67d4c26b82` rather than `pypi`, because we used
`IndexUrl::Url` rather than `IndexUrl::from`.
2024-09-16 23:33:32 +00:00
Charlie Marsh 5f2e536925
Add support for `--only-dev` to `uv sync` and `uv export` (#7367)
## Summary

Closes https://github.com/astral-sh/uv/issues/7255.

Closes https://github.com/astral-sh/uv/issues/6472.
2024-09-16 20:06:20 +00:00