Commit Graph

2161 Commits

Author SHA1 Message Date
Charlie Marsh 22795f85bc
Flatten ORs and ANDs in marker construction (#4260)
## Summary

If we're ORing an OR, we should just append rather than nesting in
another OR.

In my branch, this let us simplify:

```
python_version < '3.10' or python_version > '3.12' or (python_version < '3.8' or python_version > '3.12')
```

To:

```
python_version < '3.10' or python_version > '3.12
```
2024-06-11 21:44:49 -04:00
Charlie Marsh 44041bccd2
Bump version to v0.2.11 (#4258) 2024-06-11 20:47:25 -04:00
Charlie Marsh 44592681a0
Represent build tag as `u64` (#4253)
## Summary

The build tags in this case are like, e.g., `202206090410`. That's
larger than a `u32`, so we're rejecting the wheel. In theory build tags
could be even larger, but we already use `u64` for version segment so I
think it's fine to keep that constraint here.

I'm going to look into surfacing these errors separately.

Closes https://github.com/astral-sh/uv/issues/4252.

## Test Plan

`cargo run pip install monailabel`
2024-06-11 21:40:08 +00:00
Charlie Marsh 6dae1920af
Make missing `METADATA` file a recoverable error (#4247)
## Summary

I don't have a great way to test it, but this makes the error described
in https://github.com/astral-sh/uv/issues/4246 an incompatibility rather
than a fatal error.

Closes https://github.com/astral-sh/uv/issues/4246.
2024-06-11 19:49:38 +00:00
Zanie Blue 8cfe202e4e
Improve output when an older toolchain version is already installed (#4248)
e.g.

```
❯ uv toolchain install
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
A toolchain is already installed. Use `uv toolchain install <request>` to install a specific toolchain
```

instead of

```
❯ uv toolchain install
Using latest Python version
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
Already installed at /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.19-macos-aarch64-none
```
2024-06-11 19:49:15 +00:00
Charlie Marsh dce913c542
Warn when 'requires-python' does not include a lower bound (#4234)
## Summary

Closes https://github.com/astral-sh/uv/issues/4089.
2024-06-11 18:50:05 +00:00
Charlie Marsh b3a99d9ff9
Allow direct URLs for dev dependencies (#4233)
## Summary

Ensures that they're included in the lookahead resolver.

Closes https://github.com/astral-sh/uv/issues/4230.
2024-06-11 15:30:34 +00:00
Ibraheem Ahmed 3ea5a2a65d
De-duplicate resolution environment logic (#4232)
## Summary

Cleans up some duplicated logic in `pip install/sync/compile`. Will also
be needed for `uv add`.
2024-06-11 11:13:30 -04:00
Ibraheem Ahmed a75ed22a2f
Fix outdated comment (#4229)
## Summary

Looks like this wasn't updated at some point. I think it's referring to
`UnresolvedRequirement`?
2024-06-11 16:06:47 +02:00
Ibraheem Ahmed eefa9e62fc
Initial implementation of `uv add` and `uv remove` (#4193)
## Summary

Basic implementation of `uv add` and `uv remove` that supports writing
PEP508 requirements to `project.dependencies`.

First step for https://github.com/astral-sh/uv/issues/3959 and
https://github.com/astral-sh/uv/issues/3960.
2024-06-11 09:21:28 -04:00
Nyakku Shigure 60431ce78c
Replace `toolchain fetch` with `toolchain install` (#4228)
## Summary

Something that looks like it was forgotten to replace in #4164.

## Test Plan

Run `cargo run toolchain install` should display the warning: ``warning:
`uv toolchain install` is experimental and may change without warning.``
2024-06-11 08:00:33 -04:00
konsti 44833801b3
Support locking relative paths (#4205)
By splitting `path` into a lockable, relative (or absolute) and an
absolute installable path and by splitting between urls and paths by
dist type, we can store relative paths in the lockfile.
2024-06-11 11:58:03 +00:00
Charlie Marsh 33cf47182f
Migrate lock errors to `thiserror` (#4225)
## Summary

Do we prefer this?
2024-06-11 07:40:48 -04:00
Charlie Marsh 656fc427b9
Add support for local directories with `--index-url` (#4226)
## Summary

Closes #4078.
2024-06-10 22:27:04 -04:00
Zanie Blue f296ef08d6
Refactor project interpreter request for `requires-python` specifiers (#4216)
Refactor following #4214 to avoid parsing the specifiers again
2024-06-10 20:32:40 -05:00
samypr100 68abf85f0d
feat: mTLS support (#4171)
## Summary

Closes https://github.com/astral-sh/uv/issues/3626

This adds mTLS support to uv via the standard env var `SSL_CLIENT_CERT`.

## Test Plan

Tested locally using a [nginx proxy to
pypi](https://github.com/hauntsaninja/nginx_pypi_cache) using my own
self-signed ca + certs + client certs generated via
[mkcert](https://github.com/FiloSottile/mkcert). Used this proxy with
both uv and pip to make sure we have feature partity in mTLS
functionality.
2024-06-10 20:11:35 -05:00
Zanie Blue 5f37395f45
Allow version specifiers to be used in Python version requests (#4214)
In service of https://github.com/astral-sh/uv/issues/4212 but this is
user-facing e.g. Python discovery will support version specifiers
everywhere now.

Closes https://github.com/astral-sh/uv/issues/4212
2024-06-10 18:20:09 -05:00
Charlie Marsh 10e0abc9b1
Create temporary environments in dedicated cache bucket (#4223)
## Summary

We may choose to persist these eventually, but for now, it's useful to
have them colocated with the cache, and in their own dedicated bucket
(so, at the very least, we can keep track of the use-cases).

Closes https://github.com/astral-sh/uv/issues/4219.
2024-06-10 23:17:35 +00:00
Zanie Blue b5c9003883
Fix `pip_compile::missing_venv` test (#4224)
A merge kerfuffle from #4222  and #4218 

Now we fail because we genuinely can't find any interpreters since tests
contexts are isolated by default. I'll improve the error message and
maybe add another test case once `main` is fixed.
2024-06-10 22:39:25 +00:00
Zanie Blue 546e23bb1d
Update test context to avoid discovery of external Pythons (#4222)
By setting the test search path to an empty path, we avoid accidentally
pulling interpreters from the system during a test case.

Cherry-picked from https://github.com/astral-sh/uv/pull/4214
2024-06-10 17:26:53 -05:00
Zanie Blue 98d1ea6bb0
Improve handling of missing interpreters during discovery (#4218)
Cherry-picked from https://github.com/astral-sh/uv/pull/4214

The first commit gets us some context on an IO error during queries:

Previously:

```
failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

Now:

```
Failed to query Python interpreter
    Caused by: failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

but really we shouldn't attempt to query a missing interpreter during
discovery anyway, so we improve handling of that too.
2024-06-10 22:26:34 +00:00
Charlie Marsh 0c1dcb797a
Remove usages of verbatim URL in URL resolver (#4221)
## Summary

Should be no behavior changes, but one piece of technical debt I noticed
left over in the URL resolver. We already have structured paths, so we
shouldn't need to compare verbatim URLs.
2024-06-10 21:55:48 +00:00
Zanie Blue fd52fe74ce
Update the `Toolchain::find_requested` API to take a parsed request (#4215)
Pulled out of https://github.com/astral-sh/uv/pull/4206, need this for
#4214
2024-06-10 20:07:45 +00:00
Charlie Marsh d7e10bb4a2
Remove project dependency from `sync` (#4211)
## Summary

This is architecturally desirable because it means syncing is
independent from the project (e.g., the `pyproject.toml` and friends).
2024-06-10 19:23:37 +00:00
Charlie Marsh e31604e38b
Always install as editable when duplicates are requested (#4208)
## Summary

If the user requests a package as both editable and non-editable, the
editable now "wins".

Previously, `pip install -e . .` would install as editable. However,
`pip install -e . -r requirements.txt` would _not_ if `requirements.txt`
contained `.`, because we ignored `editable` when deduplicating and the
order of iteration was just dependent on internals.

Closes https://github.com/astral-sh/uv/issues/4053.
2024-06-10 15:02:17 -04:00
Zanie Blue 06a0fc65d4
Improve `uv toolchain list` implementation (#4203)
Amends #4163 with review from Jane, thank you!

No behavior changes.
2024-06-10 17:49:17 +00:00
Charlie Marsh 652c1126d3
Avoid crash for `XDG_CONFIG_HOME=/dev/null` (#4200)
## Summary

Closes https://github.com/astral-sh/uv/issues/4199.

## Test Plan

`XDG_CONFIG_HOME=/dev/null cargo run venv`
2024-06-10 17:36:29 +00:00
Zanie Blue 907727cb1b
Bump version to 0.2.10 (#4201) 2024-06-10 11:40:19 -05:00
Zanie Blue 14d535f384
Add `uv toolchain install` (#4164)
Adds a command (following #4163) to download and install specific
toolchains. While we fetch toolchains on demand, this is useful for,
e.g., pre-downloading a toolchain in a Docker image build.

~I kind of think we should call this `install` instead of `fetch`~ I
changed the name from `fetch` to `install`.
2024-06-10 10:56:08 -05:00
konsti 90947a933d
Recreate project environment if `--python` or `requires-python` doesn't match (#3945)
Fixes #4131
Fixes #3895
2024-06-10 14:36:14 +00:00
Charlie Marsh 125a4b220e
Improve static metadata extraction for Poetry projects (#4182)
## Summary

Adds handling for a few cases to improve interoperability with Poetry:

- If the `project` schema is invalid, we now raise a hard error, rather
than treating the metadata as dynamic and then falling back to the build
backend. This could cause problems, I'm not sure. It's stricter than
before.
- If the project contains `tool.poetry` but omits
`project.dependencies`, we now treat it as dynamic. We could go even
further and treat _any_ Poetry project as dynamic, but then we'd be
ignoring user-declared dependencies, which is also confusing.

Closes https://github.com/astral-sh/uv/issues/4142.
2024-06-10 10:26:38 -04:00
Zanie Blue c6da4f15b7
Add `uv toolchain list` (#4163)
Adds the `uv toolchain` namespace and a `list` command to get us
started.

```
❯ cargo run -q -- toolchain list
warning: `uv toolchain list` is experimental and may change without warning.
3.8.12   (cpython-3.8.12-macos-aarch64-none)
3.8.13   (cpython-3.8.13-macos-aarch64-none)
3.8.14   (cpython-3.8.14-macos-aarch64-none)
3.8.15   (cpython-3.8.15-macos-aarch64-none)
3.8.16   (cpython-3.8.16-macos-aarch64-none)
3.8.17   (cpython-3.8.17-macos-aarch64-none)
3.8.18   (cpython-3.8.18-macos-aarch64-none)
3.8.18   (cpython-3.8.18-macos-aarch64-none)
3.8.19   (cpython-3.8.19-macos-aarch64-none)
3.9.2    (cpython-3.9.2-macos-aarch64-none)
3.9.3    (cpython-3.9.3-macos-aarch64-none)
3.9.4    (cpython-3.9.4-macos-aarch64-none)
3.9.5    (cpython-3.9.5-macos-aarch64-none)
3.9.6    (cpython-3.9.6-macos-aarch64-none)
3.9.7    (cpython-3.9.7-macos-aarch64-none)
3.9.10   (cpython-3.9.10-macos-aarch64-none)
3.9.11   (cpython-3.9.11-macos-aarch64-none)
3.9.12   (cpython-3.9.12-macos-aarch64-none)
3.9.13   (cpython-3.9.13-macos-aarch64-none)
3.9.14   (cpython-3.9.14-macos-aarch64-none)
3.9.15   (cpython-3.9.15-macos-aarch64-none)
3.9.16   (cpython-3.9.16-macos-aarch64-none)
3.9.17   (cpython-3.9.17-macos-aarch64-none)
3.9.18   (cpython-3.9.18-macos-aarch64-none)
3.9.19   (cpython-3.9.19-macos-aarch64-none)
3.10.0   (cpython-3.10.0-macos-aarch64-none)
3.10.2   (cpython-3.10.2-macos-aarch64-none)
3.10.3   (cpython-3.10.3-macos-aarch64-none)
3.10.4   (cpython-3.10.4-macos-aarch64-none)
3.10.5   (cpython-3.10.5-macos-aarch64-none)
3.10.6   (cpython-3.10.6-macos-aarch64-none)
3.10.7   (cpython-3.10.7-macos-aarch64-none)
3.10.8   (cpython-3.10.8-macos-aarch64-none)
3.10.9   (cpython-3.10.9-macos-aarch64-none)
3.10.11  (cpython-3.10.11-macos-aarch64-none)
3.10.12  (cpython-3.10.12-macos-aarch64-none)
3.10.13  (cpython-3.10.13-macos-aarch64-none)
3.10.14  (cpython-3.10.14-macos-aarch64-none)
3.11.1   (cpython-3.11.1-macos-aarch64-none)
3.11.3   (cpython-3.11.3-macos-aarch64-none)
3.11.4   (cpython-3.11.4-macos-aarch64-none)
3.11.5   (cpython-3.11.5-macos-aarch64-none)
3.11.6   (cpython-3.11.6-macos-aarch64-none)
3.11.7   (cpython-3.11.7-macos-aarch64-none)
3.11.8   (cpython-3.11.8-macos-aarch64-none)
3.11.9   (cpython-3.11.9-macos-aarch64-none)
3.12.0   (cpython-3.12.0-macos-aarch64-none)
3.12.1   (cpython-3.12.1-macos-aarch64-none)
3.12.2   (cpython-3.12.2-macos-aarch64-none)
3.12.3   (cpython-3.12.3-macos-aarch64-none)
```

Closes https://github.com/astral-sh/uv/issues/4189
2024-06-10 09:22:00 -05:00
Zanie Blue a2e6aaa0ff
Fetch managed toolchains in `uv run` (#4143) 2024-06-10 14:20:44 +00:00
Zanie Blue 45df889fe4
Implement `Toolchain::find_or_fetch` and use in `uv venv --preview` (#4138)
Extends https://github.com/astral-sh/uv/pull/4121
Part of #2607 

Adds support for managed toolchain fetching to `uv venv`, e.g.

```
❯ cargo run -q -- venv --python 3.9.18 --preview -v
DEBUG Searching for Python 3.9.18 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/zb/Library/Application Support/uv/toolchains`
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Found CPython 3.9.6 at `/usr/bin/python3` (search path)
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Requested Python not found, checking for available download...
DEBUG Using registry request timeout of 30s
INFO Fetching requested toolchain...
DEBUG Downloading https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst to temporary location /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp
DEBUG Extracting cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst
DEBUG Moving /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp/python to /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none
Using Python 3.9.18 interpreter at: /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
INFO Removing existing directory
Activate with: source .venv/bin/activate
```

The preview flag is required. The fetch is performed if we can't find an
interpreter that satisfies the request. Once fetched, the toolchain will
be available for later invocations that include the `--preview` flag.
There will be follow-ups to improve toolchain management in general,
there is still outstanding work from the initial implementation.
2024-06-10 14:10:45 +00:00
Charlie Marsh 04c4da4e65
Fix `>=` to `==` typo in `is_contained_by` docs (#4196) 2024-06-10 13:28:33 +00:00
Charlie Marsh 72bc739a64
Remove `PubGrubPackage` dependency from `ResolutionGraph` (#4168)
## Summary

Similar to how we abstracted the dependencies into
`ResolutionDependencyNames`, I think it makes sense to abstract the base
packages into a `ResolutionPackage`. This also avoids leaking details
about the various `PubGrubPackage` enum variants to `ResolutionGraph`.
2024-06-10 12:50:32 +00:00
konsti 18b40b0c7d
Don't panic with invalid wheel source (#4191)
Remove the panic when there is an invalid wheel source, instead surface
the error. This error can only occur when manually editing the lock
file, but since it's an external file, we should error and not panic.

This change is helpful since the method needs to be able to error for
relative path support.
2024-06-10 08:44:36 -04:00
Charlie Marsh 763e2d2e84
Add markers to edges rather than distributions (#4166)
## Summary

We've debated this a bit but the thing that tipped me over the edge is
https://github.com/astral-sh/uv/issues/4157. As-is, there's no way to
represent "a package should be installed, but the extra should only be
installed conditionally based on the markers", because the markers sit
on the _distribution_. By placing the markers on the edge, we can now
represent scenarios that weren't previously representable.

Closes https://github.com/astral-sh/uv/issues/4137.
Closes https://github.com/astral-sh/uv/issues/4125.
Closes https://github.com/astral-sh/uv/issues/4157.
2024-06-10 08:40:51 -04:00
Charlie Marsh 5269a0dba8
Ignore tags in universal resolution (#4174)
## Summary

If a package lacks a source distribution, and we can't find a compatible
wheel for the current platform, we need to just _assume_ that the
package will have a valid wheel on all platforms on which it's
requested; if not, we raise an error at install time.

It's possible that we can be smarter about this over time. For example,
if the package was requested _only_ for macOS, we could verify that
there's at least one macOS-compatible wheel. See the linked issue for
more details.

Closes https://github.com/astral-sh/uv/issues/4139.
2024-06-10 08:38:21 -04:00
Charlie Marsh 4125cb26b8
Add `pyproject.toml` to CLI help (#4181) 2024-06-09 22:48:39 -04:00
Charlie Marsh e7c573cfcb
Fix existing typos and enable `typos` in CI (#4184) 2024-06-10 01:50:54 +00:00
Charlie Marsh 04717e1a92
Respect `--find-links` in `lock` and `sync` (#4183)
## Summary

We already respect this command-line argument, I just forgot to wire it
up to the resolver.
2024-06-09 21:42:52 -04:00
Charlie Marsh 72859d8f9b
Move PEP 440 and PEP 508 parsing out of TOML deserialization (#4176)
## Summary

If we want more granular control over how these errors are handled, then
we need to move them out of the TOML deserialization.

No actual behavior changes here.

Part of https://github.com/astral-sh/uv/issues/4142.
2024-06-10 00:09:39 +00:00
Charlie Marsh a761047df9
Reduce visibility of `pyproject.toml` fields (#4175) 2024-06-09 23:57:43 +00:00
Charlie Marsh 8ae5c2aee3
Skip version iteration for non-base packages (#4167) 2024-06-08 18:46:08 +00:00
Charlie Marsh ab0f8afe1f
Remove version from graph edge (#4165)
## Summary

We're actually not using the edge data, so lets remove it.
2024-06-08 17:39:02 +00:00
Zanie Blue 7c07ca8090
Fix test snapshot for `lock_conditional_dependency_extra` (#4159)
Introduced in #4156 and is failing on main
2024-06-08 12:04:44 -05:00
Charlie Marsh 477c860c9b
Add incorrect conditional-extra test (#4156) 2024-06-07 21:24:35 -04:00
Charlie Marsh eb239ff640
Cap `Requires-Python` comparisons at the patch version (#4150)
## Summary

See the long comment inline. I think this is debatable but probably
right for now. The other options have their own problems, but there are
a few alternate ideas in the comment.

Closes https://github.com/astral-sh/uv/issues/4132.
2024-06-08 01:22:57 +00:00
Charlie Marsh ac1ddf5e4e
Allow transitive URLs via recursive extras (#4155)
## Summary

Closes https://github.com/astral-sh/uv/issues/4152.
2024-06-08 01:10:18 +00:00
Andrew Gallant c46fa74e65
make universal resolver fork only when markers are disjoint (#4135)
The basic idea here is to make it so forking can only ever result in a
resolution that, for a particular marker environment, will only install
at most one version of a package. We can guarantee this by ensuring we
only fork on conflicting dependency specifications only when their
corresponding markers are completely disjoint. If they aren't, then
resolution _must_ find a single version of the package in the
intersection of the two dependency specifications.

A test for this case has been added to packse here:
https://github.com/astral-sh/packse/pull/182. Previously, that test
would result in a resolution with two different unconditional versions
of the same package. With this change, resolution fails (as it should).

A commit-by-commit review should be helpful here, since the first commit
is a refactor to make the second commit a bit more digestible.
2024-06-07 23:40:55 +00:00
Charlie Marsh 0db1bf4df7
Avoid pre-fetching for unbounded minimum versions (#4149)
## Summary

I think we should be able to model PubGrub such that this isn't
necessary (at least for the case described in the issue), but for now,
let's just avoid attempting to build very old distributions in
prefetching.

Closes https://github.com/astral-sh/uv/issues/4136.
2024-06-07 22:05:14 +00:00
Charlie Marsh d7cc622d6c
Accept `file://` URLs for `requirements.txt` et all references (#4145)
## Summary

Closes https://github.com/astral-sh/uv/issues/4124.
2024-06-07 22:03:08 +00:00
Zanie Blue e3b274413d
Do not create a virtual environment when locking (#4147)
Closes https://github.com/astral-sh/uv/issues/4141
2024-06-07 16:56:45 -05:00
Zanie Blue f9ea304be4
Drop "registry" prefix from request timeout log (#4144)
We use this base client for more than registry requests
2024-06-07 16:56:32 -05:00
Zanie Blue 13f03e9d3f
Reduce `uv-toolchain` discovery API to `Toolchain` (#4148)
Drops `find_toolchain`, `find_best_toolchain`, etc. in favor of
`Toolchain::find_...`

We can change this in the future, but there should only be one "right"
way to do it not two redundant ways in the public interface.
2024-06-07 16:56:25 -05:00
Zanie Blue 365bcfce1a
Update `init_environment` to use `Toolchain::find_default` API (#4146)
In service of https://github.com/astral-sh/uv/pull/4148
2024-06-07 16:16:58 -05:00
Ibraheem Ahmed 7232c53718
Simplify marker expressions in lockfile (#4066)
## Summary

Simplify and normalize marker expressions in the lockfile. Right now
this does a simple analysis by only looking at related operators at the
same level of precedence. I think anything more complex would be out of
scope.

Resolves https://github.com/astral-sh/uv/issues/4002.
2024-06-07 16:14:24 -04:00
Charlie Marsh bcfe88dfdc
Track `Markers` via a PubGrub package variant (#4123)
## Summary

This PR adds a lowering similar to that seen in
https://github.com/astral-sh/uv/pull/3100, but this time, for markers.
Like `PubGrubPackageInner::Extra`, we now have
`PubGrubPackageInner::Marker`. The dependencies of the `Marker` are
`PubGrubPackageInner::Package` with and without the marker.

As an example of why this is useful: assume we have `urllib3>=1.22.0` as
a direct dependency. Later, we see `urllib3 ; python_version > '3.7'` as
a transitive dependency. As-is, we might (for some reason) pick a very
old version of `urllib3` to satisfy `urllib3 ; python_version > '3.7'`,
then attempt to fetch its dependencies, which could even involve
building a very old version of `urllib3 ; python_version > '3.7'`. Once
we fetch the dependencies, we would see that `urllib3` at the same
version is _also_ a dependency (because we tack it on). In the new
scheme though, as soon as we "choose" the very old version of `urllib3 ;
python_version > '3.7'`, we'd then see that `urllib3` (the base package)
is also a dependency; so we see a conflict before we even fetch the
dependencies of the old variant.

With this, I can successfully resolve the case in #4099.

Closes https://github.com/astral-sh/uv/issues/4099.
2024-06-07 19:57:02 +00:00
Zanie Blue 0f4f3b4714
Update `uv run` and `uv tool run` to use `Toolchain::find` (#4134)
Extends https://github.com/astral-sh/uv/pull/4121
2024-06-07 19:28:59 +00:00
Zanie Blue 53035d65a1
Refactor `uv-toolchain` types (#4121)
Extends #4120 
Part of #2607 

There should be no behavior changes here. Restructures the discovery API
to be focused on a toolchain first perspective in preparation for
exposing a `find_or_fetch` method for toolchains in
https://github.com/astral-sh/uv/pull/4138.
2024-06-07 14:20:28 -05:00
Zanie Blue 325982c418
Rename `uv-interpreter` crate to `uv-toolchain` (#4120)
In preparation for managed toolchains #2607, just renames the crate to
something broader.

See #4121 and https://github.com/astral-sh/uv/pull/4138 to see the final
intent.
2024-06-07 13:59:14 -05:00
Charlie Marsh 2803a8c475
Omit URL dependencies from pre-release hints (#4140)
## Summary

Closes https://github.com/astral-sh/uv/issues/4127.
2024-06-07 18:43:50 +00:00
Charlie Marsh 7d1b7b99d9
Rename `Dependency.id` to `Dependency.distribution_id` (#4114)
## Summary

I think this makes clearer that the `Dependency.id` is not an identifier
for the dependency itself.

No functional changes.
2024-06-07 18:28:54 +00:00
Charlie Marsh fa10679275
Add extra and dev dependency validation to lockfile (#4112)
## Summary

Closes https://github.com/astral-sh/uv/issues/4106.

Closes https://github.com/astral-sh/uv/issues/4115.
2024-06-07 14:18:31 -04:00
Charlie Marsh d3651c13f0
Set `--dev` to default for `uv run` and `uv sync` (#4118)
## Summary

Closes https://github.com/astral-sh/uv/issues/4117.
2024-06-07 00:06:05 +00:00
Zanie Blue dcf70a1f29
Include non-standard ports in keyring host queries (#4061)
Partially addresses https://github.com/astral-sh/uv/issues/4056

We were incorrectly omitting the port from requests to `keyring` when
falling back to a realm/host query, e.g. `localhost` was used instead of
`localhost:1234`. We still won't include "standard" ports like `80` for
an HTTP request.
2024-06-06 19:02:47 -05:00
Zanie Blue cb7d6245ae
Use `PythonEnvironment` API in `uv pip compile` (#4030)
Same as https://github.com/astral-sh/uv/pull/4029
2024-06-06 18:54:40 -05:00
Charlie Marsh cc7c780523
Remove PubGrub dependency from `uv` (#4116)
## Summary

Encapsulates more of the details are `Requires-Python` and PubGrub.

Closes https://github.com/astral-sh/uv/issues/4110.
2024-06-06 23:45:58 +00:00
Charlie Marsh 52bdee2e85
Add support for `--prefix` (#4085)
## Summary

Closes #3076.
2024-06-06 16:15:28 -04:00
Charlie Marsh 677a7f157b
Avoid showing dev hints for Python requirements (#4111)
Closes https://github.com/astral-sh/uv/issues/4096.
2024-06-06 19:58:30 +00:00
konsti e4e2590076
Use union of `requires-python` in workspace (#4041)
Follow-up to #4016.

This exposes `Range` and `PubGrubSpecifier` from outside the resolver to
use pubgrub's union creating a dependency edge we don't really want.
2024-06-06 19:21:02 +00:00
konsti a6f53e2aa4
Lock all packages in workspace (#4016)
When creating a lockfile, lock the combined dependencies for all
packages in a workspace. This make the lockfile independent of where you
are in the workspace.

Fixes #3983
2024-06-06 19:09:44 +00:00
Charlie Marsh e9fc99e622
Bump version to v0.2.9 (#4107) 2024-06-06 14:21:33 -04:00
Charlie Marsh 31bb01f0be
Ignore upper-bounds on `Requires-Python` (#4086)
## Summary

This PR modifies our `Requires-Python` handling to treat
`Requires-Python` as a lower bound. There's extensive discussion around
this in https://github.com/astral-sh/uv/issues/4022 and the references
linked therein. I think it's an experiment worth trying. Even in my own
small projects, I'm running into issues whereby I'm being "forced" to
add a `<4` upper bound to my `Requires-Python` due to these caps.

Separately, we should explore adding a mechanism that's distinct from
`Requires-Python` to enable users to declare a supported range for
locking.

Closes https://github.com/astral-sh/uv/issues/4022.
2024-06-06 13:52:57 -04:00
Charlie Marsh 30e73a60de
Avoid enforcing distribution ID uniqueness for extras (#4104)
## Summary

The condition enforced here isn't quite right. The same dependency can
appear multiple times, as long as the extra is different.

Closes https://github.com/astral-sh/uv/issues/4101.
2024-06-06 15:21:31 +00:00
Charlie Marsh 8798e91dd5
Avoid extra-only filtering for constraints (#4095)
## Summary

The "only include if relevant for the extra" filtering should _not_ be
applied to constraints. Otherwise, we'd only constrain when the extra
was included in the constraints file itself, which is incorrect.

Closes https://github.com/astral-sh/uv/issues/4091.
2024-06-06 13:58:46 +00:00
konsti 39f8978920
Fix logging in uv-dev (#4093)
Set the correct default logging directive and filtering in uv-dev. See
#4090.
2024-06-06 13:06:57 +00:00
Charlie Marsh fa2b6a28bc
Prefix sys fields with `sys_` consistently on Interpreter (#4084)
## Summary

This is more consistent across the struct _and_ opens up space for
`--prefix`.
2024-06-06 02:11:21 +00:00
Charlie Marsh 120148f0a1
Remove integration tests from `uv-resolver` (#4083)
## Summary

I don't think it's worth maintaining this separate test harness for ~18
tests, when they can all be tested in the `uv` package itself. Let's
drop the maintenance burden.
2024-06-06 01:48:42 +00:00
Charlie Marsh 0acae9bd9c
Add support for development dependencies (#4036)
## Summary

Externally, development dependencies are currently structured as a flat
list of PEP 580-compatible requirements:

```toml
[tool.uv]
dev-dependencies = ["werkzeug"]
```

When locking, we lock all development dependencies; when syncing, users
can provide `--dev`.

Internally, though, we model them as dependency groups, similar to
Poetry, PDM, and [PEP 735](https://peps.python.org/pep-0735). This
enables us to change out the user-facing frontend without changing the
internal implementation, once we've decided how these should be exposed
to users.

A few important decisions encoded in the implementation (which we can
change later):

1. Groups are enabled globally, for all dependencies. This differs from
extras, which are enabled on a per-requirement basis. Note, however,
that we'll only discover groups for uv-enabled packages anyway.
2. Installing a group requires installing the base package. We rely on
this in PubGrub to ensure that we resolve to the same version (even
though we only expect groups to come from workspace dependencies anyway,
which are unique). But anyway, that's encoded in the resolver right now,
just as it is for extras.
2024-06-06 01:40:17 +00:00
Charlie Marsh a81fb92ee6
Respect existing `.egg-link` files in site packages (#4082)
## Summary

As with other `.egg-info` and `.egg-link` distributions, it's easy to
support _existing_ `.egg-link` files. Like pip, we refuse to uninstall
these, since there's no way to know which files are part of the
distribution.

Closes https://github.com/astral-sh/uv/issues/4059.

## Test Plan

Verify that `vtk` is included here, which is installed as a `.egg-link`
file:

```
> conda create -c conda-forge -n uv-test python h5py vtk pyside6 cftime psutil

> cargo run pip freeze --python /opt/homebrew/Caskroom/miniforge/base/envs/uv-test/bin/python
aiohttp @ file:///Users/runner/miniforge3/conda-bld/aiohttp_1713964997382/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
cftime @ file:///Users/runner/miniforge3/conda-bld/cftime_1715919201099/work
frozenlist @ file:///Users/runner/miniforge3/conda-bld/frozenlist_1702645558715/work
h5py @ file:///Users/runner/miniforge3/conda-bld/h5py_1715968397721/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1713279365350/work
loguru @ file:///Users/runner/miniforge3/conda-bld/loguru_1695547410953/work
msgpack @ file:///Users/runner/miniforge3/conda-bld/msgpack-python_1715670632250/work
multidict @ file:///Users/runner/miniforge3/conda-bld/multidict_1707040780513/work
numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1707225421156/work/dist/numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl
pip==24.0
psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1705722460205/work
pyside6==6.7.1
setuptools==70.0.0
shiboken6==6.7.1
vtk==9.2.6
wheel==0.43.0
wslink @ file:///home/conda/feedstock_root/build_artifacts/wslink_1716591560747/work
yarl @ file:///Users/runner/miniforge3/conda-bld/yarl_1705508643525/work
```
2024-06-06 01:00:00 +00:00
Zanie Blue e6817bd997
Add regression test for `VIRTUAL_ENV` and `uv venv` interaction (#4074)
For https://github.com/astral-sh/uv/pull/4073

Demonstrated to fail without those changes.
2024-06-06 00:28:45 +00:00
Zanie Blue e48b9cd94b
Bump version to 0.2.8 (#4076) 2024-06-05 21:09:05 +00:00
Zanie Blue bacde427f1
Fix `uv venv` handling when `VIRTUAL_ENV` refers to an non-existant environment (#4073)
Closes https://github.com/astral-sh/uv/issues/4072

This was an accidental change in
https://github.com/astral-sh/uv/pull/4029 in which I had updated the
pull request to support virtual environments as requested in review and
forgot to revert it.

Separately, we shouldn't fail if `VIRTUAL_ENV` points to an empty
directory and `SystemPython::Allowed` is used, will address that
separately.
2024-06-05 20:50:28 +00:00
Charlie Marsh e5f95186de
Default to current Python minor if `Requires-Python` is absent (#4070)
## Summary

If `Requires-Python` is omitted in `uv lock` or `uv run`, we now warn
and default to `>=` the current minor version.

Closes https://github.com/astral-sh/uv/issues/4050.
2024-06-05 20:45:50 +00:00
Charlie Marsh 0b6d5b37dc
Enforce `Requires-Python` when syncing (#4068)
## Summary

Ensures that we raise if the user attempts to use a Python version that
wasn't included in the locked range.

Closes https://github.com/astral-sh/uv/issues/4052.
2024-06-05 16:22:08 -04:00
Charlie Marsh 642cef0dad
Track supported Python range in lockfile (#4065)
## Summary

This PR adds the `Requires-Python` range to the user's lockfile. This
will enable us to validate it when installing.

For now, we repeat the `Requires-Python` back to the user;
alternatively, though, we could detect the supported Python range
automatically.

See: https://github.com/astral-sh/uv/issues/4052
2024-06-05 16:21:59 -04:00
konsti 8596525d97
Update pubgrub to new `add_incompatibility_from_dependencies` (#4062)
We had previously changed the signature of
`DependencyProvider::get_dependencies` to return an iterator instead of
a hashmap to avoid the conversion cost from our dependencies `Vec` to
the pubgrub's hashmap. These changes are difficult to make in pubgrub
since they complicate the public api. But we don't actually use
`DependencyProvider::get_dependencies`, so we rolled those
customizations back in https://github.com/pubgrub-rs/pubgrub/pull/226
and instead opted to change only the internal
`add_incompatibility_from_dependencies` method that we exposed in our
fork. This aligns us closer with upstream, removes the design questions
about `DependencyProvider` from our concerns and reduces our diff (not
counting the github action) to +36 -12.
2024-06-05 20:46:00 +02:00
konsti 9f1b592fca
Retry flaky `compile_invalid_pyc_invalidation_mode` test (#4043)
Retry the flaky `compile_invalid_pyc_invalidation_mode` if it fails. I
don't understand why this happening in the first place (we have code
that should catch those cases, but also those cases shouldn't be
happening at all) and this is terrible hack, but it fixes the test
flakes.

Fixes #2672
2024-06-05 20:37:17 +02:00
Charlie Marsh a241f148de
Bump version to v0.2.7 (#4060) 2024-06-05 14:25:25 -04:00
Charlie Marsh 191f9556b7
Avoid building packages with dynamic versions (#4058)
## Summary

This PR separates "gathering the requirements" from the rest of the
metadata (e.g., version), which isn't required when installing a
package's _dependencies_ (as opposed to installing the package itself).
It thus ensures that we don't need to build a package when a static
`pyproject.toml` is provided in `pip compile`.

Closes https://github.com/astral-sh/uv/issues/4040.
2024-06-05 18:11:58 +00:00
Charlie Marsh a0173760f1
Remove unused PEP 621 error (#4057) 2024-06-05 17:45:42 +00:00
Charlie Marsh c97427d530
Reduce visibility of `lowering` (#4055)
## Summary

This makes `lowering.rs` internal to the metadata package.
2024-06-05 13:39:37 -04:00
Charlie Marsh 34f847bb68
Remove unused `Requirements` struct (#4054) 2024-06-05 17:14:57 +00:00
Zanie Blue db3c36dbcc
Use `PythonEnvironment` API in `uv venv` (#4029)
There's no reason to be reaching into the lower-level `find_interpreter`
manually here.
2024-06-05 11:49:29 -05:00
konsti b05a39c735
Store explicit project on workspace member (#4048)
We know that `[project]` must exist for each workspace member, so we can
store it directly and avoid going through the `.and_then()` when we need
to access it. This requires cloning the struct due to lack of
self-referential structs. An alternative would taking the `Project` from
`PyProjectToml` instead, but this could be confusing when passing the
`PyProjectToml` around.
2024-06-05 12:48:19 -04:00
Charlie Marsh ae9610104a
Avoid dropping `pip sync` requirements with markers (#4051)
## Summary

Thankfully this is pretty rare since `pip sync` is usually run on `pip
compile` output, and `pip compile` never outputs markers.

Closes https://github.com/astral-sh/uv/issues/4044
2024-06-05 16:05:46 +00:00
Zanie Blue b0d1fc85a9
Use standard path filters for `uv venv` tests (#4046)
Over in #4045 the existing filters were insufficient. We should use the
same strategy we use for our standard `TestContext` instead.
2024-06-05 10:25:10 -05:00
Zanie Blue f2dc08b242
Drop `git` dependency from `pip_entrypoints` test (#4049)
There's a release with the changes we want test coverage for now
2024-06-05 17:22:32 +02:00
Zanie Blue 91c5ac44ac
Add `ProvidedPath` to interpreter sources considered when `UV_TEST_PYTHON_PATH` is set (#4027)
Just a theory, but I presume this would close
https://github.com/astral-sh/uv/issues/4023.

I'm not sure how else we'd end up in this case.
2024-06-05 09:52:28 -05:00
Zanie Blue 5db7f7b3ac
Move `uv-interpreter::managed::*` modules into crate top-level (#4020)
Should be no functional changes, just some file renames in preparation
for exposing toolchain management.
2024-06-05 07:27:30 -05:00
Zanie Blue 2be93c0b2c
Display selected interpreter sources during not selected error (#4026)
This should help us narrow down the weird error in
https://github.com/astral-sh/uv/issues/4023
2024-06-05 07:26:56 -05:00
Charlie Marsh 6d148da0ab
Preserve fragments when applying verbatim redirects (#4038)
## Summary

`echo
"git+https://github.com/pypa/sample-namespace-packages.git#subdirectory=pkg_resources/pkg_a"
| cargo run pip compile -` now resolves to a compliant URL.

Closes https://github.com/astral-sh/uv/issues/4037.
2024-06-05 03:53:23 +00:00
Charlie Marsh 27e5bfe397
Remove unnecessary requirements conversions (#4035) 2024-06-05 01:52:16 +00:00
Charlie Marsh 8de3e38b94
Read all extras directly from lockfile (#4033)
## Summary
We now pass `ExtrasSpecification` to the lock routine.
2024-06-05 01:03:55 +00:00
samypr100 1b3200b2af
feat: support `NO_COLOR` and `FORCE_COLOR` env vars (#3979)
## Summary

Closes #3955

Adds explicit support to `NO_COLOR` and `FORCE_COLOR` via
GlobalSettings.

The order, per specs is now `NO_COLOR` > `FORCE_COLOR` > `color`.

This PR is a backup plan pending rust-cli/anstyle#192.

## Test Plan

Tested all cases locally for now; I didn't see existing tests for
GlobalSettings parsing.
2024-06-04 17:00:42 -04:00
Charlie Marsh 57ea55d218
Add a hint for `Requires-Python` (#4021)
## Summary

As requested in the originating PR.
2024-06-04 16:52:15 -04:00
Charlie Marsh 8eea470d49
Bias towards local directories for bare editable requirements (#3995)
## Summary

Given `install -e dagster`, we need to assume that the user meant
`install -e ./dagster`, even though `install dagster` should _not_ be
treated as `install ./dagster`. I suspect pip will change this in the
future (since `pip install dagster` does _not_ meant `pip install
./dagster`) but for now it's what users expect.

Closes https://github.com/astral-sh/uv/issues/3994.
2024-06-04 19:37:05 +00:00
Charlie Marsh 420333a40e
Avoid 'are incompatible' for singular bounded versions (#4003)
## Summary

Not sure if this is worth the complexity, but it does read better.
2024-06-04 19:17:36 +00:00
Andrew Gallant 365ca637c7 uv/tests: add new universal tests
This updates to packse 0.3.17. There are a few incidental changes in
places, but the main addition here are the tests for universal
resolutions.
2024-06-04 14:24:59 -04:00
Andrew Gallant 459966a132
uv/tests: update packse tests (#4015)
This is just the result of running

    ./scripts/sync_scenarios.sh

From the root of the `uv` repository.

When I initially ran this, it produced some tests with snapshots that
weren't being updated. It turned out this was because the tests weren't
running, as they were gated behind the `python-patch` feature. In this
commit, we add `python-patch` to our `cargo insta` command, which should
update all relevant snapshots.

There are still some superfluous updates as a result of a spell checker
being run on generated files, but
2024-06-04 13:56:20 -04:00
Andrew Gallant 5c30b39fe3
uv-resolver: normalize marker expressions (#4017)
This is a quick fix for some flaky tests where the output in the lock
file isn't stable because marker expressions can be combined in a
non-deterministic order.

I believe there is ongoing work to simplify marker expressions which
will help here, but I think some kind of normalization is still
ultimately needed to guarantee consistent output.

I first noticed the flaky test in:
https://github.com/astral-sh/uv/pull/4015
2024-06-04 13:45:54 -04:00
Ibraheem Ahmed 3b8f3a7f0d
Avoid work-stealing in bytecode compilation (#4004)
## Summary

Avoid using work-stealing Tokio workers for bytecode compilation,
favoring instead dedicated threads. Tokio's work-stealing does not
really benefit us because we're spawning Python workers and scheduling
tasks ourselves — we don't want Tokio to re-balance our workers. Because
we're doing scheduling ourselves and compilation is a primarily
compute-bound task, we can also create dedicated runtimes for each
worker and avoid some synchronization overhead.

This is part of a general desire to avoid relying on Tokio's
work-stealing scheduler and be smarter about our workload. In this case
we already had the custom scheduler in place, Tokio was just getting in
the way (though the overhead is very minor).

## Test Plan

This improves performance by ~5% on my machine.

```
$ hyperfine --warmup 1 --prepare "target/profiling/uv-dev clear-compile .venv" "target/profiling/uv-dev compile .venv" "target/profiling/uv-dev-dedicated compile .venv"
Benchmark 1: target/profiling/uv-dev compile .venv
  Time (mean ± σ):      1.279 s ±  0.011 s    [User: 13.803 s, System: 2.998 s]
  Range (min … max):    1.261 s …  1.296 s    10 runs
 
Benchmark 2: target/profiling/uv-dev-dedicated compile .venv
  Time (mean ± σ):      1.220 s ±  0.021 s    [User: 13.997 s, System: 3.330 s]
  Range (min … max):    1.198 s …  1.272 s    10 runs

Summary
  target/profiling/uv-dev-dedicated compile .venv ran
    1.05 ± 0.02 times faster than target/profiling/uv-dev compile .venv

$ hyperfine --warmup 1 --prepare "target/profiling/uv-dev clear-compile .venv" "target/profiling/uv-dev compile .venv" "target/profiling/uv-dev-dedicated compile .venv"
Benchmark 1: target/profiling/uv-dev compile .venv
  Time (mean ± σ):      3.631 s ±  0.078 s    [User: 47.205 s, System: 4.996 s]
  Range (min … max):    3.564 s …  3.832 s    10 runs
 
Benchmark 2: target/profiling/uv-dev-dedicated compile .venv
  Time (mean ± σ):      3.521 s ±  0.024 s    [User: 48.201 s, System: 5.392 s]
  Range (min … max):    3.484 s …  3.566 s    10 runs
 
Summary
  target/profiling/uv-dev-dedicated compile .venv ran
    1.03 ± 0.02 times faster than target/profiling/uv-dev compile .venv
```
2024-06-04 10:48:23 -04:00
Charlie Marsh 6afb659c9a
Respect `Requires-Python` in universal resolution (#3998)
## Summary

Closes #3982.
2024-06-04 13:56:08 +00:00
konsti 63c84ed4a6
Log transient network request failures (#3933)
We retry several kinds of network request failures, but it's often
unclear whether a request was retried or not
(https://github.com/astral-sh/uv/issues/3514#issuecomment-2105485773).
This PR adds a small intermediary layer that logs all transient request
failures, adding the `DEBUG Transient request failure` lines:

```
DEBUG Searching for Python interpreter in virtual environments
DEBUG Found CPython 3.12.3 at `/home/konsti/projects/uv/.venv/bin/python3` (active virtual environment)
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: tqdm
DEBUG Using registry request timeout of 30s
DEBUG Solving with target Python version 3.12.3
DEBUG Adding direct dependency: tqdm*
DEBUG No cache entry for: https://pypi.org/simple/tqdm/
DEBUG Transient request failure for https://pypi.org/simple/tqdm/, retrying: Request error: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
DEBUG Transient request failure for https://pypi.org/simple/tqdm/, retrying: Request error: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
DEBUG Transient request failure for https://pypi.org/simple/tqdm/, retrying: Request error: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
DEBUG Transient request failure for https://pypi.org/simple/tqdm/, retrying: Request error: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
error: Could not connect, are you offline?
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
```

I decided for multi-line logging to show the complete error trace since
only `Transient request failure for https://pypi.org/simple/tqdm/,
retrying: Request error: error sending request for url
(https://pypi.org/simple/tqdm/)` doesn't tell you the actual problem (a
dns error).

Note that running with `-v` will not show messages about retry backoff
timing, but running with `RUST_LOG=debug` now shows a complete picture:

```
DEBUG starting new connection: https://pypi.org/
DEBUG resolving host="pypi.org"
DEBUG Transient request failure for https://pypi.org/simple/tqdm/, retrying: Request error: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: error sending request for url (https://pypi.org/simple/tqdm/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Name or service not known
  Caused by: failed to lookup address information: Name or service not known
WARN Retry attempt #2. Sleeping 528.728192ms before the next attempt
```

Fixes #3572
2024-06-04 15:39:16 +02:00
Di-Is 41ec302557
Fix a bug where no warning is output when parsing of workspace settings fails. (#4014)
## Summary

See #4013

`uv pip ...` command loads workspace settings from pyproject.toml and
uv.toml.

Although a warning is implemented to output a warning when parsing
fails, it is not actually output.


https://github.com/astral-sh/uv/blob/main/crates/uv-workspace/src/workspace.rs#L38-L61

The reason is that the flag to display warnings is enabled after loading
the workspace settings.

This PR turns on the warning output flag before loading the workspace.

## Test Plan

pyproject.toml for test

```toml
[project]
name = "sample"
version = "0.0.0"
dependencies = ["ruff"]

[tool.uv.pip]
# originally string type.
index-url = 1
```

command output (before modification)
```bash
uv pip compile pyproject.toml 

Resolved 1 package in 383ms
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml
ruff==0.4.7
    # via sample (pyproject.toml)
```

command output (after modification)

```bash
uv pip compile pyproject.toml

warning: Failed to parse `pyproject.toml`: TOML parse error at line 7, column 13
  |
7 | index-url = true
  |             ^^^^
invalid type: boolean `true`, expected a string

Resolved 1 package in 107ms
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml
ruff==0.4.7
    # via sample (pyproject.toml)
```
2024-06-04 09:21:19 -04:00
konsti da7d5549a3
Don't copy gitignored files in workspace tests (#4012)
The workspace test directories can be used both in tests and directly
for developing/debugging. In the latter, we shouldn't copy the venv and
the lockfile when running tests. Using the ignore crate over manual
recursion we exclude those files.
2024-06-04 12:58:07 +00:00
konsti 36f7fa3917
`impl TryFrom<&VersionSpecifiers> for PubGrubSpecifier` (#4010)
Add a missing utility conversion method from PEP 440 versions specifiers
to a pubgrub range.
2024-06-04 08:00:07 +00:00
Charlie Marsh 77e93157fb
Make target Python version an optional field (#4000)
## Summary

Instead of checking if the target and installed version are the same, we
model the data such that the target version is only present if it was
specified by the user. This also means that we correctly say "requested
version" even if the two happen to be the same.
2024-06-03 22:37:15 +00:00
Charlie Marsh 037e7e345c
Remove unused `dummy` method (#4001) 2024-06-03 22:28:45 +00:00
Charlie Marsh b27d36baee
Remove some unnecessary `Interpreter` clones (#3999) 2024-06-03 22:09:42 +00:00
Charlie Marsh ef43bcb233
Remove Python from available versions (#3996)
## Summary

I believe this is no longer necessary. Part of the problem here is that
we can't _know_ the full set of available Python versions, especially
once we start resolving against a `Requires-Python` rather than a fixed
set of two versions.
2024-06-03 20:11:45 +00:00
Charlie Marsh 10cd6b94c9
Normalize extras in lockfile (#3958)
## Summary

Previously, when we locked something like `flask[dotenv]`, we created
two separate distributions in the lockfile: one for `flask`, which
included the base dependencies, and one for `flask[dotenv]`, which
included the base dependencies _and_ the `dotenv` dependencies. This was
easy to implement, but it meant that we were duplicating all of the
distribution files for every extra, and duplicating all of the base
dependencies for every extra.

This PR normalizes the data such that we now have one entry per
distribution (i.e., `ExtraName` was removed from `DistributionId`), with
an optional dependencies table with an entry per extra, like:

```toml
[[distribution]]
name = "project"
version = "0.1.0"
source = "editable+file://[TEMP_DIR]/"
sdist = { url = "file://[TEMP_DIR]/" }

[[distribution.dependencies]]
name = "anyio"
version = "3.7.0"
source = "registry+https://pypi.org/simple"

[distribution.optional-dependencies]

[[distribution.optional-dependencies.test]]
name = "iniconfig"
version = "2.0.0"
source = "registry+https://pypi.org/simple"
```

This requires a bit more work upfront, because we now need to merge
multiple packages from the `PetGraph` representation when creating the
lockfile.

Closes https://github.com/astral-sh/uv/issues/3916.
2024-06-03 19:00:35 +00:00
Charlie Marsh 362b00cc12
Remove need to return Python version in `get_dependencies` (#3993)
## Summary

Once we use a _range_ rather than a precise version, it won't actually
make sense to return a version here. It's no longer required, so I'm
removing it.
2024-06-03 18:42:38 +00:00
Charlie Marsh 1a60368ce4
Use `PubGrubPython` type in Python incompatibility reporting (#3992)
## Summary

Rather than re-testing compatibility, I think we can just rely on the
types directly.
2024-06-03 14:32:22 -04:00
Charlie Marsh a589ad5066
Bump version to v0.2.6 (#3991) 2024-06-03 17:40:28 +00:00
Ibraheem Ahmed 1ffe18d861
Avoid race condition in `OnceMap` (#3987)
## Summary

Fixes a race condition in `OnceMap::wait_blocking` where the inserted
value could potentially be missed, leading to a deadlock. Fairly certain
this will resolve https://github.com/astral-sh/uv/issues/3724.
2024-06-03 12:25:58 -04:00
Charlie Marsh 29ea5d5d9a
Fix reference to `--python-version` patch behavior (#3989)
## Summary

We changed this at some point but didn't update the documentation.
2024-06-03 15:34:12 +00:00
Charlie Marsh 650638fa52
Remove unstable uv lock from pip interface (#3970)
## Summary

I think we can start using `uv lock` and `uv sync` to test this instead.
2024-06-03 15:14:11 +00:00
Andrew Gallant fbf562d5a8
uv {lock,sync}: propagate index URLs to registry client (#3986)
Otherwise the `uv lock` command wasn't respecting the index URL option.

This is a follow-up to #3984, and I believe should now allow #3970 to be
merged.
2024-06-03 14:38:36 +00:00
Charlie Marsh a4c73fd07e
Extract index arguments into their own Clap group (#3985)
## Summary

Just removes a lot of repeated documentation. We can expand this
strategy in the future.
2024-06-03 14:27:35 +00:00
Charlie Marsh da1782808a
Add index URL parameters to Project CLI (#3984)
## Summary

For now, this exists to facilitate testing. We may remove it in the
future in favor of other APIs.
2024-06-03 10:17:41 -04:00
Tim de Jager 1b1600c40e
feat: add back the use of extra env vars to the build dispatch (#3981)
Seems like a recent Pull removed this, couldn't directly find out which.
I'm adding it back as we rely on this API, and I do not see another way
of accessing this, or am I mistaken?

Thanks!
2024-06-03 09:13:44 -04:00
Di-Is 5c776939d2
Add override namespace to pyproject.toml/uv.toml (#3839)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

See #3834 .

This PR adds a new namespace, `override-dependencies`, to
pyproject.toml/uv.toml.
This namespace assumes that the dependencies you want to override are
written in the form of `requirements.txt`.


a example of pyproject.toml
```toml
[project]
name = "example"
version = "0.0.0"
dependencies = [
  "flask==3.0.0"
]

[tool.uv]
override-dependencies = [
  "werkzeug==2.3.0"
]
```

This will improve usability by allowing you to override dependencies
without having to specify the --override option when running `uv pip
compile/install`.

## Test Plan

added test to `crates/uv/tests/pip_compile.rs`.

---------

Co-authored-by: konstin <konstin@mailbox.org>
2024-06-03 12:15:51 +02:00
Charlie Marsh a23ca5bab7
Use universal resolution in `uv lock` (#3969)
## Summary

Wires up the optional markers in resolution, and adds
respecting-the-markers to `Lock:: to_resolution`.
2024-06-02 21:33:18 -04:00
Charlie Marsh c500b78936
Avoid re-adding solutions to forked state (#3967)
## Summary

Running a resolution that required forking was failing due to breaking
an invariant in PubGrub. It looks like we were adding the same
incompatibility multiple times, or something like that. The issue
appears to be that when forking, we modify the current state, then clone
it as the "next state", then push to the "forked states" -- but that
means we're cloning the _modified_ state.

This PR changes the order of operations such that we clone, then modify.
It shouldn't introduce any additional clones though.
2024-06-02 17:58:25 -04:00
konsti 01d1a39c21
Add `uv run --package` (#3864)
Add a `--package` option that allows switching the current project in
the workspace. Wherever you are in a workspace, you should be able to
run with any other project as root. This is the uv equivalent of `cargo
run -p`.

I don't love the `--package` name, esp. since `-p` is already taken and
in general to many things start with p already.

Part of this change is moving the workspace discovery of
`ProjectWorkspace` to `Workspace` itself.

## Usage

In albatross-virtual-workspace:

```console
$ uv venv
$ uv run --preview --package bird-feeder python -c "import albatross"
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds
Built 2 editables in 167ms
Resolved 5 packages in 4ms
Installed 5 packages in 1ms
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds)
 + sniffio==1.3.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'albatross'
$ uv venv
$ uv run --preview --package albatross python -c "import albatross"
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/albatross
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds
Built 3 editables in 173ms
Resolved 7 packages in 6ms
Installed 7 packages in 1ms
 + albatross==0.1.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/albatross)
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds)
 + sniffio==1.3.1
 + tqdm==4.66.4
```

In albatross-root-workspace:

```console
$ uv venv
$ uv run --preview --package bird-feeder python -c "import albatross"
  Using Python 3.12.3 interpreter at: /home/konsti/.local/bin/python3
  Creating virtualenv at: .venv
  Activate with: source .venv/bin/activate
      Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
       Running `/home/konsti/projects/uv/target/debug/uv run --preview --package bird-feeder python -c 'import albatross'`
     Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder
     Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds                                              Built 2 editables in 161ms
  Resolved 5 packages in 4ms
  Installed 5 packages in 1ms
   + anyio==4.4.0
   + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder)
   + idna==3.6
   + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds)
   + sniffio==1.3.1
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named 'albatross'
$ uv venv
$ cargo run run --preview --package albatross python -c "import albatross"
Using Python 3.12.3 interpreter at: /home/konsti/.local/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `/home/konsti/projects/uv/target/debug/uv run --preview --package albatross python -c 'import albatross'`
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds
Built 3 editables in 168ms
Resolved 7 packages in 5ms
Installed 7 packages in 1ms
 + albatross==0.1.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace)
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds)
 + sniffio==1.3.1
 + tqdm==4.66.4
```
2024-06-02 21:42:14 +00:00
Matthieu Darbois 0d0308c531
fix: add missing ppc64le alias for powerpc64le (#3963)
## Summary

Same as #3899 but for ppc64le, there were no tests added there so I
wouldn't know where to begin to do so.

Using image docker image `quay.io/pypa/manylinux2014_ppc64le` (uv 0.2.4)

```
[root@e3ff544d1337 ~]# python3.12 -V
Python 3.12.3
[root@e3ff544d1337 ~]# RUST_LOG="trace" uv venv py312 --python python3.12
DEBUG Searching for Python 3.12 in search path
TRACE Searching PATH for executables: python3.12, python3, python
TRACE Checking `PATH` directory for interpreters: /opt/rh/devtoolset-10/root/usr/bin
TRACE Checking `PATH` directory for interpreters: /usr/local/sbin
TRACE Checking `PATH` directory for interpreters: /usr/local/bin
TRACE Found possible Python executable: /usr/local/bin/python3.12
TRACE Querying interpreter executable at /usr/local/bin/python3.12
TRACE Querying Python at `/usr/local/bin/python3.12` did not return the expected data
unknown variant `ppc64le`, expected one of `aarch64`, `armv6l`, `armv7l`, `powerpc64le`, `powerpc64`, `x86`, `x86_64`, `s390x`
--- stdout:
{"result": "success", "markers": {"implementation_name": "cpython", "implementation_version": "3.12.3", "os_name": "posix", "platform_machine": "ppc64le", "platform_python_implementation": "CPython", "platform_release": "6.6.26-linuxkit", "platform_system": "Linux", "platform_version": "#1 SMP Sat Apr 27 04:13:19 UTC 2024", "python_full_version": "3.12.3", "python_version": "3.12", "sys_platform": "linux"}, "base_prefix": "/opt/_internal/cpython-3.12.3", "base_exec_prefix": "/opt/_internal/cpython-3.12.3", "prefix": "/opt/_internal/cpython-3.12.3", "base_executable": "/usr/local/bin/python3.12", "sys_executable": "/usr/local/bin/python3.12", "sys_path": ["/root/.cache/uv/.tmpBnM4PN", "/opt/_internal/cpython-3.12.3/lib/python312.zip", "/opt/_internal/cpython-3.12.3/lib/python3.12", "/opt/_internal/cpython-3.12.3/lib/python3.12/lib-dynload", "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages"], "stdlib": "/opt/_internal/cpython-3.12.3/lib/python3.12", "scheme": {"platlib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "purelib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "include": "/opt/_internal/cpython-3.12.3/include/python3.12", "scripts": "/opt/_internal/cpython-3.12.3/bin", "data": "/opt/_internal/cpython-3.12.3"}, "virtualenv": {"purelib": "lib/python3.12/site-packages", "platlib": "lib/python3.12/site-packages", "include": "include/site/python3.12", "scripts": "bin", "data": ""}, "platform": {"os": {"name": "manylinux", "major": 2, "minor": 17}, "arch": "ppc64le"}, "gil_disabled": false, "pointer_size": "64"}
--- stderr:

---
TRACE Skipping bad interpreter at /usr/local/bin/python3.12
TRACE Checking `PATH` directory for interpreters: /usr/sbin
TRACE Checking `PATH` directory for interpreters: /usr/bin
TRACE Found possible Python executable: /usr/bin/python
TRACE Querying interpreter executable at /usr/bin/python
TRACE Can't use Python at `/usr/bin/python`
TRACE Skipping bad interpreter at /usr/bin/python
TRACE Checking `PATH` directory for interpreters: /sbin
TRACE Checking `PATH` directory for interpreters: /bin
TRACE Found possible Python executable: /bin/python
TRACE Querying interpreter executable at /bin/python
TRACE Can't use Python at `/bin/python`
TRACE Skipping bad interpreter at /bin/python
  × No interpreter found for Python 3.12 in search path
```

---------

Co-authored-by: Charlie Marsh <crmarsh416@gmail.com>
2024-06-02 13:15:24 -04:00
Charlie Marsh 11324646cb
Remove some `anyhow` usages (#3962) 2024-06-01 20:11:23 +00:00
Charlie Marsh a70e33d947
Move reference check into `uv-git` (#3961) 2024-06-01 16:02:25 -04:00
Charlie Marsh c04a95e037
Respect resolved Git SHAs in `uv lock` (#3956)
## Summary

This PR ensures that if a lockfile already contains a resolved reference
(e.g., you locked with `main` previously, and it locked to a specific
commit), and you run `uv lock`, we use the same SHA, even if it's not
the latest SHA for that tag. This avoids upgrading Git dependencies
without `--upgrade`.

Closes #3920.
2024-06-01 12:40:11 +00:00
Charlie Marsh b7d77c04cc
Add Git resolver in lieu of static hash map (#3954)
## Summary

This PR removes the static resolver map:

```rust
static RESOLVED_GIT_REFS: Lazy<Mutex<FxHashMap<RepositoryReference, GitSha>>> =
    Lazy::new(Mutex::default);
```

With a `GitResolver` struct that we now pass around on the
`BuildContext`. There should be no behavior changes here; it's purely an
internal refactor with an eye towards making it cleaner for us to
"pre-populate" the list of resolved SHAs.
2024-05-31 22:44:42 -04:00
Charlie Marsh a0652921fc
Don't enforce that requirements files are "files" (#3947)
## Summary

This was just an opportunistic guard but it broke some use-cases.

Closes #3944.
2024-05-31 21:32:36 +00:00
Charlie Marsh 7b7da803bb
Unify editable and unnamed URL parsing (#3946)
## Summary

This will help prevent bugs like #3934 by unifying the implementations
for editables and non-editable unnamed requirements. Specifically, both
of these now go through the same parsing paths and use the same struct
representations (with the exception that the editable flag is flipped in
the first case):

```
-e ./foo/bar
./foo/bar
```

We also now support PEP 508 in editable URLs. It turns out this is just
a limitation in pip, so it's correct to support it. For example, this
now works:

```
-e black[d] @ file://${PROJECT_ROOT}/scripts/packages/black_editable
```

Closes #3941.

Closes #3942.
2024-05-31 21:08:00 +00:00
Charlie Marsh 8c11f99fdf
Discard fragments when parsing unnamed URLs (#3940)
## Summary

Closes https://github.com/astral-sh/uv/issues/3934.
2024-05-31 13:54:15 +00:00
konsti 72b1642232
Move metadata into its own file (#3939)
Move `Metadata`, `MetadataLoweringError` and `ArchiveMetadata` into
their own file `metadata.rs` in `uv-distribution`, moving it out from
`lib.rs`. No functional changes.
2024-05-31 15:24:10 +02:00
konsti 3c074142f5
Re-add lowering unit tests (#3935)
Re-add the lowering unit tests removed in #3904. This also adds a
`stop_discovery_at` feature to avoid running actual workspace discovery.
2024-05-31 12:17:49 +00:00
konsti 9bb0679618
Fix nightly cfg checker warnings (#3932)
Fixes these two warnings on nightly:

```
warning: unexpected `cfg` condition name: `codspeed`
 --> crates/bench/src/lib.rs:5:15
  |
5 |     #[cfg(not(codspeed))]
  |               ^^^^^^^^ help: found config with similar value: `feature = "codspeed"`
  |
  = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows`
  = help: consider using a Cargo feature instead
  = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
           [lints.rust]
           unexpected_cfgs = { level = "warn", check-cfg = ['cfg(codspeed)'] }
  = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(codspeed)");` to the top of the `build.rs`
  = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
  = note: `#[warn(unexpected_cfgs)]` on by default

warning: unexpected `cfg` condition name: `codspeed`
 --> crates/bench/src/lib.rs:8:11
  |
8 |     #[cfg(codspeed)]
  |           ^^^^^^^^ help: found config with similar value: `feature = "codspeed"`
  |
  = help: consider using a Cargo feature instead
  = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
           [lints.rust]
           unexpected_cfgs = { level = "warn", check-cfg = ['cfg(codspeed)'] }
  = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(codspeed)");` to the top of the `build.rs`
  = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
```

```
warning: unexpected `cfg` condition value: `unix`
 --> crates/uv-extract/src/tar.rs:6:16
  |
6 | #[cfg_attr(not(target_os = "unix"), allow(dead_code))]
  |                ^^^^^^^^^^^^^^^^^^
  |
  = note: expected values for `target_os` are: `aix`, `android`, `cuda`, `dragonfly`, `emscripten`, `espidf`, `freebsd`, `fuchsia`, `haiku`, `hermit`, `horizon`, `hurd`, `illumos`, `ios`, `l4re`, `linux`, `macos`, `netbsd`, `none`, `nto`, `openbsd`, `psp`, `redox`, `solaris`, `solid_asp3`, `teeos`, `tvos`, `uefi`, `unknown`, `visionos`, `vita`, `vxworks`, `wasi`, `watchos`, and `windows` and 2 more
  = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
  = note: requested on the command line with `-W unexpected-cfgs`
```

<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
2024-05-31 09:35:52 +00:00
konsti 081f20c53e
Add support for `tool.uv` into distribution building (#3904)
With the change, we remove the special casing of workspace dependencies
and resolve `tool.uv` for all git and directory distributions. This
gives us support for non-editable workspace dependencies and path
dependencies in other workspaces. It removes a lot of special casing
around workspaces. These changes are the groundwork for supporting
`tool.uv` with dynamic metadata.

The basis for this change is moving `Requirement` from
`distribution-types` to `pypi-types` and the lowering logic from
`uv-requirements` to `uv-distribution`. This changes should be split out
in separate PRs.

I've included an example workspace `albatross-root-workspace2` where
`bird-feeder` depends on `a` from another workspace `ab`. There's a
bunch of failing tests and regressed error messages that still need
fixing. It does fix the audited package count for the workspace tests.
2024-05-31 02:42:03 +00:00
Charlie Marsh 09f55482a0
Remove some unused `pub use` exports (#3930) 2024-05-30 22:26:52 -04:00
Charlie Marsh a14fe2f6c7
Avoid filtering preferences by `--reinstall` (#3929)
## Summary

In general, it's not quite right to filter preferences by `--reinstall`
-- we still want to respect existing versions, we just don't want to
respect _installed_ versions. But now that the installed versions and
preferences are decoupled, we can remove this (`--reinstall` is enforced
on the installed versions via the `Exclusions` struct that we pass to
the resolver).

While I was here, I also cleaned up the lockfile preference code to
better match the structure for `requirements.txt`.
2024-05-30 20:19:08 +00:00
Charlie Marsh 438b5c61d0
Remove installed packages from preferences (#3928)
## Summary

I believe that this is not necessary, as the installer packages are
already considered in `CandidateSelector::get_preferred`.

Firstly, note that we never pass both non-empty installed packages _and_
non-empty preferences (the installer routines pass site packages and no
preferences; the resolver routines pass no site packages but lockfile
preferences).

However, in general, if you look at `CandidateSelector::get_preferred`,
and consider what's changing, we now skip the `if let Some(version) =
preferences.version(package_name)` case for installed packages. But we
then check installed packages within that `if`, and in the `else`. So it
seems like we'll still return them in either case?

The only behavior change is in the case that you have multiple versions
of a package installed. Previously, we'd respect one of them, because
`Preferences` takes the last winner (it's a hash map, so we just replace
the package key with the last version we see); but in installed
packages, we always ignore distributions with multiple versions, since
it's indicative of a broken environment. That's a fine change IMO. We
could change `CandidateSelector::get_preferred` to support this if we
wanted to.
2024-05-30 20:09:31 +00:00
Ibraheem Ahmed 261aa2c70a
Port all git functionality to use git CLI (#3833)
## Summary

We currently rely on libgit2 for most git-related functionality.
However, libgit2 has long-standing performance issues, as well as lags
significantly behind git in terms of new features. For these reasons we
now use the git CLI by default for fetching repositories
(https://github.com/astral-sh/uv/pull/1781). This PR completely drops
libgit2 in favor of the git CLI for all git-related functionality, which
should allow us to use features such as partial clones and sparse
checkouts in the future for performance.

There is also a lot of technical debt in the current git code as it's
mostly taken from Cargo. Switching to the git CLI *vastly* simplifies
the `uv-git` codebase.

Eventually we might want to look into switching to
[`gitoxide`](https://github.com/Byron/gitoxide), but it's currently too
immature for our use case.
2024-05-30 15:28:48 -04:00
Ibraheem Ahmed 85183c1c36
Custom lock-file serialization (#3909)
## Summary

This PR changes the lock-file format to use inline tables for wheels and
source distributions, which currently use separate tables that make the
file harder to follow.

```diff
[[distribution]]
name = "typing-extensions"
version = "4.10.0"
source = "registry+https://pypi.org/simple"

- [distribution.sdist]
- url = "0d26ce356c7c323176620b7b483e44/typing_extensions-4.10.0.tar.gz"
- hash = "sha256:b0abd7c89e8fb96f98db18d86106ff1d90ab692004eb746cf6eda2682f91b3cb"
- size = 77558
-
- [[distribution.wheel]]
- url = "dc04a3ea60b9bc8074b5d6b7ab90bb/typing_extensions-4.10.0-py3-none-any.whl"
- hash = "sha256:69b1a937c3a517342112fb4c6df7e72fc39a38e7891a5730ed4985b5214b5475"
- size = 33926

+ sdist = { url = "0d26ce356c7c323176620b7b483e44/typing_extensions-4.10.0.tar.gz", hash = "sha256:b0abd7c89e8fb96f98db18d86106ff1d90ab692004eb746cf6eda2682f91b3cb", size = 77558 }
+ wheel = [{ url = "dc04a3ea60b9bc8074b5d6b7ab90bb/typing_extensions-4.10.0-py3-none-any.whl", hash = "sha256:69b1a937c3a517342112fb4c6df7e72fc39a38e7891a5730ed4985b5214b5475", size = 33926 }]
```

The downside is that the inline-tables end up quite long and TOML
doesn't support line breaks in inline tables, yet.

Part of https://github.com/astral-sh/uv/issues/3611.
2024-05-30 19:08:29 +00:00
Andrew Gallant d3b7d800ea uv-resolver: fix perf regression
We significantly regressed performance in some cases because we were
cloning the resolver state one more time than we needed to. That doesn't
sound like a lot, but in the case where there are no forks, it implies
we were cloning the state for every `get_dependencies` called when we
shouldn't have been cloning it at all.

Avoiding the clone results in somewhat tortured code. This can probably
be refactored by moving bits out to a helper routine, but that also
seemed non-trivial. So we let this suffice for now.
2024-05-30 14:23:14 -04:00
Andrew Gallant 17c043536b uv-resolver: thread markers through the resolver and into the lock file
This addresses the lack of marker support in prior commits.
Specifically, we add them as a new field to `AnnotatedDist`, and from
there, they get added to a `Distribution` in a `Lock`.
2024-05-30 14:23:14 -04:00
Andrew Gallant f865406ab4 uv-resolver: implement merging of forked resolutions
This commit is a pretty invasive change that implements the merging
of resolutions created by each fork of the resolver.

The main idea here is that each `SolveState` is converted into a
`Resolution` (a new type) and stored on the heap after its fork
completes. When all forks complete, they are all merged into a single
`Resolution`. This `Resolution` is then used to build a `ResolutionGraph`.

Construction of `ResolutionGraph` mostly stays the same (despite the
gnarly diff due to an indent change) with one exception: the code to
extract dependency edges out of PubGrub's state has been moved to
`SolveState::into_resolution`. The idea here is that once a fork
completes, we extract what we need from the PubGrub state and then
throw it away. We store these edges in our own intermediate type which
is then converted into petgraph edges in the `ResolutionGraph`
constructor.

One interesting change we make here is that our edge
data is now a `Version` instead of a `Range<Version>`. I don't think
`Range<Version>` was actually being used anywhere, so this seems okay?
In any case, I think `Version` here is correct because a resolution
corresponds to specific dependencies of each package. Moreover, I didn't
see an easy way to make things work with `Range<Version>`. Notably,
since we no longer have the guarantee that there is only one version of
each package, we need to use `(PackageName, Version)` instead of just
`PackageName` for inverted lookups in `ResolutionGraph::from_state`.

Finally, the main resolver loop itself is changed a bit to track all
forked resolutions and then merge them at the end.

Note that we don't really have any dealings with markers in this commit.
We'll get to that in a subsequent commit.
2024-05-30 14:23:14 -04:00
Andrew Gallant 9e977aa1be uv-resolver: slightly simplify ResolutionGraph::from_state
This changes the constructor to just take an `InMemoryIndex`
directly instead of the constituent parts. No real reason other
than it seems a little simpler.
2024-05-30 14:23:14 -04:00
Andrew Gallant 6f76a66510 uv-resolver: implement basic resolver forking
There are still some TODOs/FIXMEs here, but this makes represents a
chunk of the resolver refactoring to enable forking. We don't do any
merging of resolutions yet, so crucially, this code is broken when no
marker environment is provided. But when a marker environment is
provided, this should behave the same as a non-forking resolver. In
particular, `get_dependencies_forking` is just `get_dependencies`
whenever there's a marker environment.
2024-05-30 14:23:14 -04:00
Andrew Gallant f5f330627b pep508: make 'simplify_extras' more flexible
This makes it so we can pass any function to determine whether an extra
is always true or not.

For example, `markers.simplify_extras_with(|_| true)` will remove all
extras in any marker expression. This wasn't possible to express
(without knowing all of the marker names) using the old API, but becomes
trivial to express with a predicate function.
2024-05-30 14:23:14 -04:00
Andrew Gallant a74c2c6792 pep508: add `and` and `or` mutators to `MarkerTree`
While this could be done by callers since the representation
of `MarkerTree` is public, they are just annoying enough to do
that I think it makes sense to provide them on `MarkerTree`
itself.

These could also be improved in the future to do even more
flattening of conjunctions/disjunctions (or perhaps even
more robust simplification). But for now, some basic flattening
is good enough.

These routines will be used to combine marker expressions when
merging forked resolutions.
2024-05-30 14:23:14 -04:00
Charlie Marsh 144566907e
Use lockfile versions as resolution preferences (#3921)
## Summary

Ensures that we avoid upgrading packages unless `--upgrade` or similar
is passed.

For now, the resolver only respects these for registry distributions.

Closes https://github.com/astral-sh/uv/issues/3918.
2024-05-30 17:59:53 +00:00
Charlie Marsh 502e04200d
Remove `from_registry_built_dist2` (#3922)
## Summary

I think this was left over from a prior refactor. (The existing
`from_registry_built_dist` was unused.)
2024-05-30 01:57:19 +00:00
Charlie Marsh a1a5155d34
Move preference behavior out of resolve (#3919)
## Summary

Decoupling this behavior to give more control to clients. No change in
behavior.
2024-05-30 01:16:53 +00:00
Charlie Marsh 4859a27948
Add extra dependency annotations to lockfile and sync commands (#3913)
## Summary

This PR adds extras to the lockfile, and enables users to selectively
sync extras in `uv sync` and `uv run`. The end result here was fairly
simple, though it required a few refactors to get here. The basic idea
is that `DistributionId` now includes `extra: Option<ExtraName>`, so we
effectively treat extras as separate packages. Generating the lockfile,
and generating the resolution from the lockfile, fall out of this
naturally with no special-casing or additional changes.

The main downside here is that it bloats the lockfile significantly.
Specifically:

- We include _all_ distribution URLs and hashes for _every_ extra
variant.
- We include all dependencies for the extra variant, even though that
are dependencies of the base package.

We could normalize this representation by changing each distribution
have an `optional-dependencies` hash map that keys on extras, but we
actually don't have the information we need to create that right now
(specifically, we can't differentiate between dependencies that
_require_ the extra and dependencies on the base package).

Closes #3700.
2024-05-29 19:25:58 +00:00
Charlie Marsh 1bd5d8bc34
Include all extras when generating lockfile (#3912)
## Summary

This PR just ensures that when running `uv lock` (or `uv run`), we lock
with all extras. When we later install, we'll also _install_ with all
extras, but that will be changed in a future PR.
2024-05-29 15:08:20 -04:00
Charlie Marsh fb0dfef671
Migrate lockfile tests to `uv lock` and `uv sync` (#3914) 2024-05-29 18:00:37 +00:00
Charlie Marsh ed7f55606d
Split `requirements.txt`-style resolution distribution into its own type (#3911) 2024-05-29 16:48:28 +00:00
konsti af06544347
Remove dead code from distribution-types (#3910)
Remove some dead code. Split out from #3904
2024-05-29 12:02:42 -04:00
Charlie Marsh 19c91e7dac
Create distinct graph nodes for each package extra (#3908)
## Summary

Today, we represent each package as a single node in the graph, and
combine all the extras. This is helpful for the `requirements.txt`-style
resolution, in which we want to show each a single line for each package
with the extras combined into a single array.

This PR modifies the representation to instead use a separate node for
each (package, extra) pair. We then reduce into the previous format when
printing in the `requirements.txt`-style format, so there shouldn't be
any user-facing changes here.
2024-05-29 15:42:49 +00:00
konsti 0edb660205
Add missing i686 alias for x86 (#3899)
This alias is required for the 32-bit x86 manylinux builder.

I've also changed the reporting level for unusable python interpreter
errors to debug so it shows up with `-v`, otherwise we can't see what's
broken:

```console
# uv venv py312 --python python3.12 -vv
    0.000461s DEBUG uv_interpreter::discovery Searching for Python 3.12 in search path
  × No interpreter found for Python 3.12 in search path
```

Reproduction:

```console
$ docker run --rm -it quay.io/pypa/manylinux2014_i686
# curl -LsSf https://astral.sh/uv/install.sh | s
# source $HOME/.cargo/env
# RUST_LOG=trace uv venv py312 --python python3.12
DEBUG Searching for Python 3.12 in search path
TRACE Searching PATH for executables: python3.12, python3, python
TRACE Checking `PATH` directory for interpreters: /root/.cargo/bin
TRACE Checking `PATH` directory for interpreters: /opt/rh/devtoolset-10/root/usr/bin
TRACE Checking `PATH` directory for interpreters: /usr/local/sbin
TRACE Checking `PATH` directory for interpreters: /usr/local/bin
TRACE Found possible Python executable: /usr/local/bin/python3.12
TRACE Querying interpreter executable at /usr/local/bin/python3.12
TRACE Querying Python at `/usr/local/bin/python3.12` did not return the expected data
unknown variant `i686`, expected one of `aarch64`, `armv6l`, `armv7l`, `powerpc64le`, `powerpc64`, `x86`, `x86_64`, `s390x`
--- stdout:
{"result": "success", "markers": {"implementation_name": "cpython", "implementation_version": "3.12.3", "os_name": "posix", "platform_machine": "i686", "platform_python_implementation": "CPython", "platform_release": "6.1.0-10-amd64", "platform_system": "Linux", "platform_version": "#1 SMP PREEMPT_DYNAMIC Debian 6.1.37-1 (2023-07-03)", "python_full_version": "3.12.3", "python_version": "3.12", "sys_platform": "linux"}, "base_prefix": "/opt/_internal/cpython-3.12.3", "base_exec_prefix": "/opt/_internal/cpython-3.12.3", "prefix": "/opt/_internal/cpython-3.12.3", "base_executable": "/usr/local/bin/python3.12", "sys_executable": "/usr/local/bin/python3.12", "sys_path": ["/root/.cache/uv/.tmp7k64ZY", "/opt/_internal/cpython-3.12.3/lib/python312.zip", "/opt/_internal/cpython-3.12.3/lib/python3.12", "/opt/_internal/cpython-3.12.3/lib/python3.12/lib-dynload", "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages"], "stdlib": "/opt/_internal/cpython-3.12.3/lib/python3.12", "scheme": {"platlib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "purelib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "include": "/opt/_internal/cpython-3.12.3/include/python3.12", "scripts": "/opt/_internal/cpython-3.12.3/bin", "data": "/opt/_internal/cpython-3.12.3"}, "virtualenv": {"purelib": "lib/python3.12/site-packages", "platlib": "lib/python3.12/site-packages", "include": "include/site/python3.12", "scripts": "bin", "data": ""}, "platform": {"os": {"name": "manylinux", "major": 2, "minor": 17}, "arch": "i686"}, "gil_disabled": false, "pointer_size": "32"}
--- stderr:

---
TRACE Skipping bad interpreter at /usr/local/bin/python3.12
TRACE Checking `PATH` directory for interpreters: /usr/sbin
TRACE Checking `PATH` directory for interpreters: /usr/bin
TRACE Found possible Python executable: /usr/bin/python
TRACE Querying interpreter executable at /usr/bin/python
TRACE Can't use Python at `/usr/bin/python`
TRACE Skipping bad interpreter at /usr/bin/python
TRACE Checking `PATH` directory for interpreters: /sbin
TRACE Checking `PATH` directory for interpreters: /bin
TRACE Found possible Python executable: /bin/python
TRACE Querying interpreter executable at /bin/python
TRACE Can't use Python at `/bin/python`
TRACE Skipping bad interpreter at /bin/python
```
2024-05-29 11:31:39 -04:00
Ahmed Ilyas 1a9aa35eae
Fix workspace settings - remove deny unknown fields (#3907)
## Summary

This PR addresses an issue where `tool.uv` settings are not read if
`tool.uv.sources` or `tool.uv.workspaces` are present in the TOML file.

## Test Plan

Tested locally.
2024-05-29 11:31:07 -04:00
Charlie Marsh 42b1ba04ec
Remove unused `PetGraph` weight (#3906) 2024-05-29 15:16:25 +00:00
Charlie Marsh 2301e00863
Use lockfile in `uv run` (#3894)
## Summary

Modifies `uv run` to write and read from the lockfile, rather than
resolving the project requirements as-is on each invocation.

Closes https://github.com/astral-sh/uv/issues/3891.
2024-05-29 13:28:21 +00:00
Ibraheem Ahmed 9658485f06
Initialize multi-progress state before individual bars (#3901)
## Summary

Resolves https://github.com/astral-sh/uv/issues/3896. Adding progress
bars to the `MultiProgress` after configuring them seems to not
synchronize the required state fully.

## Test Plan

The repeated output is gone when testing locally.
2024-05-29 09:23:40 -04:00
konsti 1e1f49bfe5
Remove unnecessary `pub` in uv-build (#3900)
Expose only `SourceBuild` and the error type from `uv-build`, all other
types are internal.
2024-05-29 09:30:21 +00:00
Charlie Marsh 3461c8b585
Move extra specification into `uv-configuration` (#3897)
## Summary

I need to use this in the resolver (and it's at-home with other, similar
configuration options).
2024-05-29 04:49:57 +00:00
Charlie Marsh 2b6c24ed2d
Rename project API error (#3893) 2024-05-28 21:59:24 -04:00
Ibraheem Ahmed 038af6e658
Disable concurrent progress bars in Jupyter Notebooks (#3890)
## Summary

Resolves https://github.com/astral-sh/uv/issues/3887 by disabling the
new progress output when the `JPY_SESSION_NAME` environment variable is
detected.
2024-05-28 17:05:11 -04:00
Charlie Marsh 47db418ba2
Bump version to v0.2.5 (#3885) 2024-05-28 18:05:32 +00:00
Charlie Marsh 78c43c6a7a
Move editable discovery behind `--preview` for now (#3884)
## Summary

Just reduces the risk of shipping stable regressions for now.
2024-05-28 17:54:25 +00:00
konsti f92447772e
Support x86 windows (#3873)
Add trampolines for x86 windows.

Closes https://github.com/astral-sh/uv/issues/3660.
2024-05-28 16:07:39 +00:00
Charlie Marsh cedd18e4c6
Remove some unused `pub` functions (#3872)
## Summary

I wrote a bad Python script to find these.
2024-05-28 15:58:13 +00:00
Charlie Marsh 1fc6a59707
Remove special-casing for editable requirements (#3869)
## Summary

There are a few behavior changes in here:

- We now enforce `--require-hashes` for editables, like pip. So if you
use `--require-hashes` with an editable requirement, we'll reject it. I
could change this if it seems off.
- We now treat source tree requirements, editable or not (e.g., both `-e
./black` and `./black`) as if `--refresh` is always enabled. This
doesn't mean that we _always_ rebuild them; but if you pass
`--reinstall`, then yes, we always rebuild them. I think this is an
improvement and is close to how editables work today.

Closes #3844.

Closes #2695.
2024-05-28 15:49:34 +00:00
konsti 063a0a4384
Add context to failed `uv tool run` (#3882)
These are not covered by `fs_err` and a missing binary otherwise only
give a generic "file not found" error.
2024-05-28 14:48:54 +00:00
Tim de Jager b12b25db10
fix: remove unused `Result` in uv-cache (#3875)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
It removes the unused result when creating the Cache with the
`from_path` constructor. I don't believe it does any io operations any
more at least.
2024-05-28 09:19:08 +00:00
konsti a89e146107
Initial workspace support (#3705)
Add workspace support when using `-r <path>/pyproject.toml` or `-e
<path>` in the pip interface. It is limited to all-editable
static-metadata workspaces, and tests only include a single main
workspace, ignoring path dependencies in another workspace. This can be
considered the MVP for workspace support: You can create a workspace,
you can install from it, but some options and conveniences are still
missing. I'll file follow-up tickets (support in lockfiles, support path
deps in other workspace, #3625)

There is also support in `uv run`, but we need
https://github.com/astral-sh/uv/issues/3700 first to properly support
using different current projects in the bluejay interface, currently the
resolution and therefore the lockfile depends on the current project.
I'd do this change first (it's big enough already), then #3700, and then
add workspace support properly to bluejay.

Fixes #3404
2024-05-28 07:41:53 +00:00
Charlie Marsh 89cfecea77
Add support for `prepare_metadata_for_build_editable` hook (#3870)
## Summary

See:
https://peps.python.org/pep-0660/#prepare-metadata-for-build-editable
2024-05-27 23:21:45 +00:00
Charlie Marsh 22dbc79994
Use consistent argument ordering for `Dist` conversions (#3868) 2024-05-27 19:48:58 +00:00
Charlie Marsh 3c61eabd9d
Remove unnecessary `::try_from` (#3867)
## Summary

I didn't realize this even worked? We only define a `From` for this
conversion.
2024-05-27 15:18:20 -04:00
Tim de Jager 65b17f6e81
feat: bump pep580 and pep440 types (#3860)
This bumps the versions of pep580 and pep440 to coincide with the
crates.io versions. While not strictly the same, the new types in uv us
an `Inner` struct. Practically I've found I'm still able to use the
patched versions, as can seen from the open PR here:
https://github.com/prefix-dev/pixi/pull/1436.

Would be great if this bump can be done so we can keep combining the
types :)
2024-05-27 09:38:39 +02:00
Zanie Blue 66eb642938
Add test context utility to `uv-interpreter` (#3832)
Follow-up to https://github.com/astral-sh/uv/pull/3797 to clean up the
test isolation in `uv-interpreter`.

I still want to expose a CLI at some point like `uv python <...>` for
discovery and test from there, hopefully this will make that transition
simpler.
2024-05-27 04:17:49 +00:00
Charlie Marsh 14fa49b7ba
Move availability enums into their own module (#3858) 2024-05-27 00:12:53 -04:00
Zanie Blue 31b0ff6373
Gate discovery of managed toolchains with preview (#3835)
Prepares for merge of https://github.com/astral-sh/uv/pull/3797, gating
managed toolchain discovery with the preview flag to lower risk of
releasing.

e.g.

```
❯ cargo dev -q fetch-python         
❯ cargo run -q -- venv --python 3.11
Using Python 3.11.9 interpreter at: /opt/homebrew/opt/python@3.11/bin/python3.11
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ cargo run -q -- venv --preview --python 3.11
Using Python 3.11.7 interpreter at: /Users/zb/Library/Application Support/uv/toolchains/cpython-3.11.7-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```

We'll add automatic fetching of managed interpreters later.
2024-05-27 04:05:19 +00:00
Zanie Blue 30e780e1dd
Add persistent storage of installed toolchains (#3797)
Extends #3726 

Moves toolchain storage out of `UV_BOOTSTRAP_DIR` (`./bin`) into the
proper user data directory as defined by #3726.

Replaces `UV_BOOTSTRAP_DIR` with `UV_TOOLCHAIN_DIR` for customization.
Installed toolchains will be discovered without opt-in, but the idea is
still that these are not yet user-facing.
2024-05-27 03:54:49 +00:00
Charlie Marsh 4191c331a7
Update bundled Python URLs and add `"arm"` architecture variant (#3855)
## Summary

This also adds filtering for the ARM Pythons, since that needs some libc
changes; and it closes https://github.com/astral-sh/uv/issues/3854 by
way of adding an "arm" branch.
2024-05-26 23:33:08 -04:00
Charlie Marsh 61ed0c82ed
Remove unused `Platform` enum (#3856) 2024-05-26 23:21:56 -04:00
Ibraheem Ahmed 7dc322665c
Concurrent progress bars (#3252)
## Summary

Implements concurrent progress bars. Resolves
https://github.com/astral-sh/uv/issues/1209.

## Test Plan

https://github.com/astral-sh/uv/assets/34988408/b21bdfbb-8817-4873-a65c-16c9e8c7c460
2024-05-27 01:21:07 +00:00
Charlie Marsh 70cbc32565
Remove `tracing::log` usage (#3853) 2024-05-27 01:03:28 +00:00