Commit Graph

2161 Commits

Author SHA1 Message Date
Andrew Gallant e264637b63 uv/tests: add more fork marker tests
These were prompted by @konstin's question here:
https://github.com/astral-sh/uv/pull/4339#discussion_r1642622391
2024-06-17 09:30:37 -04:00
Andrew Gallant 6a7b56ca05 uv/tests: add fork marker packse tests 2024-06-17 09:30:37 -04:00
Andrew Gallant dbb12bcfe4 uv-resolver: fix bug in marker disjointness checking
I found this while testing the tracking of marker expressions across
resolver forks. Namely, given

    sys_platform == 'darwin' and implementation_name == 'pypy'

And:

    sys_platform == 'bar' or implementation_name == 'foo'

These should be disjoint, but the disjointness checker was reporting
them as overlapping. I fixed this by giving handling of disjunctions
higher precedence than conjunctions, although I am not 100% confident
that this is correct for all cases.
2024-06-17 09:30:37 -04:00
Andrew Gallant 407f1e370b uv-resolver: filter dependencies that can't exist in a fork
This commit adds marker expressions to our `Fork` type, which are in
turn passed down into `PubGrubDependencies::from_requirements` to filter
our any dependencies with markers that are disjoint from the fork's
marker expression.

This is necessary to avoid visiting packages in the dependency graph
that can never actually be installed. This is because when a fork is
created in the resolver, it always happens when there are two sibling
dependency specifications on a package with the same name, but with
non-overlapping marker expressions. Each fork corresponds to each
such conflicting dependency specification, and each fork assumes the
the corresponding marker expression as a pre-condition for any future
dependencies considered by it. That is, since the fork represents an
installation path that can only be taken when the corresponding
dependency specification (and its marker expression) is actually used,
it also therefore follows that the marker expression is true. Therefore,
any dependency visited in that fork with a marker expression that cannot
possibly be true when the markers of the fork are true can and ought to
be completely ignored.
2024-06-17 09:30:37 -04:00
Andrew Gallant 07db2b167f uv-resolver: document some of our intermediate data structures
There are some key invariants that I had to re-learn by reading the
code. This hopefully makes those invariants easier to discover by future
me (and others).
2024-06-17 09:30:37 -04:00
Zanie Blue 52bb9a694c
Isolate virtual environment tests from developer toolchains (#4342)
Otherwise, when testing `uv venv --preview` the default toolchain
directory will leak into the test.

I believe I've made a similar change for the standard `TestContext` in
another commit somewhere in my stack, if not I'll add it after.
2024-06-17 05:40:11 -05:00
Ibraheem Ahmed 294f0e0c41
Add support for adding/removing development dependencies (#4327)
## Summary

Support adding/removing dependencies from `tool.uv.dev-dependencies`
with `uv add/remove --dev`.

Part of https://github.com/astral-sh/uv/issues/3959.
2024-06-14 19:17:29 +00:00
Ibraheem Ahmed 042fdea087
Support unnamed requirements in `uv add` (#4326)
## Summary

Support unnamed URL requirements in `uv add`. For example, `uv add
git+https://github.com/pallets/flask`.

Part of https://github.com/astral-sh/uv/issues/3959.
2024-06-14 13:42:39 -04:00
Zanie Blue accbb9b695
Add `uv toolchain find` (#4206)
Adds a command to find a toolchain on the system. Right now, it displays
the path to the first matching toolchain. We'll probably have more rich
output in the future (after implementing `toolchain show`).

The eventual plan (separate from here) is to port all of the toolchain
discovery tests to use this command. I'll add a few tests for this
command here anyway.
2024-06-14 17:03:16 +00:00
Charlie Marsh b7fb0b445f
Use portable slash paths in lockfile (#4324)
## Summary

This would be a lightweight solution to
https://github.com/astral-sh/uv/issues/4307 that doesn't fully engage
with all the possibilities in the design space (but would unblock
cross-platform for now).
2024-06-14 09:05:14 -04:00
Charlie Marsh 74c05683bb
Add a derive macro for `Combine` (#4325)
## Summary

Saves us some boilerplate when adding settings in the future.
2024-06-14 04:53:27 +00:00
Charlie Marsh 83067c1802
Reduce some `anyhow` usages (#4323) 2024-06-14 04:15:53 +00:00
Charlie Marsh 7d9541d0f4
Add `--no-build`, `--no-build-package`, and binary variants (#4322)
## Summary

These are now supported on `uv run`, `uv lock`, `uv sync`, and `uv tool
run`.

Closes https://github.com/astral-sh/uv/issues/4297.
2024-06-14 04:05:00 +00:00
Charlie Marsh f01ab57518
Add a `--show-settings` option for configuration testing (#4304)
## Summary

The fixtures here are pretty large, but it lets us test what we actually
care about (the resolved settings) rather than inferring the resolved
settings from behavior, which I think is a big improvement.

I also broke the tests down into more granular cases.
2024-06-14 03:14:23 +00:00
Charlie Marsh 1d6d98f3a3
Make `--reinstall`, `--upgrade`, and `--refresh` shared arguments (#4319)
## Summary

Ensures that we respect these in all the relevant `uv` APIs.

Closes https://github.com/astral-sh/uv/issues/4316.
2024-06-14 01:43:18 +00:00
Charlie Marsh db84825908
Omit project name from workspace errors (#4299)
## Summary

Because the workspace member itself is part of the resolution, adding
the workspace name for the project leads to confusing errors, like:

```
❯ cargo run lock --preview
   Compiling uv v0.2.11 (/Users/crmarsh/workspace/puffin/crates/uv)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.79s
     Running `/Users/crmarsh/workspace/puffin/target/debug/uv lock --preview`
  × No solution found when resolving dependencies:
  ╰─▶ Because only albatross==0.1.0 is available and albatross==0.1.0 depends on anyio<=3, we can conclude that all versions of albatross depend on anyio<=3.
      And because bird-feeder==1.0.0 depends on anyio>=4.3.0,<5 and only bird-feeder==1.0.0 is available, we can conclude that all versions of albatross and all versions of bird-feeder are incompatible.
      And because albatross depends on albatross and bird-feeder, we can conclude that the requirements are unsatisfiable.
```

(Notice "albatross depends on albatross".)
2024-06-14 01:32:51 +00:00
Charlie Marsh cacd1a2b5a
Load configuration options from workspace root (#4295)
## Summary

In a workspace, we now read configuration from the workspace root.
Previously, we read configuration from the first `pyproject.toml` or
`uv.toml` file in path -- but in a workspace, that would often be the
_project_ rather than the workspace configuration.

We need to read configuration from the workspace root, rather than its
members, because we lock the workspace globally, so all configuration
applies to the workspace globally.

As part of this change, the `uv-workspace` crate has been renamed to
`uv-settings` and its purpose has been narrowed significantly (it no
longer discovers a workspace; instead, it just reads the settings from a
directory).

If a user has a `uv.toml` in their directory or in a parent directory
but is _not_ in a workspace, we will still respect that use-case as
before.

Closes #4249.
2024-06-14 01:26:20 +00:00
Charlie Marsh e0a389032f
Add persistent configuration for non-`pip` APIs (#4294)
## Summary

This PR introduces top-level configuration for uv, such that you can do:

```toml
[tool.uv]
index-url = "https://test.pypi.org/simple"
```

And `uv pip compile`, `uv run`, `uv tool run`, etc., will all respect
that configuration.

The settings that were escalated to the top-level remain on
`tool.uv.pip` too, but they're only respected in `uv pip` commands. If
they're specified in both places, then the `pip` settings win out.

While making this change, I also wired up some of the global options,
like `connectivity` and `native_tls`, through to all the relevant
places.

Closes #4250.
2024-06-13 20:56:38 -04:00
Zanie Blue b74de31967
Add `--force` option to `uv toolchain install` (#4313) 2024-06-13 16:43:40 -05:00
Zanie Blue 92802df223
Suggest correct command to create a virtual environment when encountering externally managed interpreters (#4314) 2024-06-13 16:43:31 -05:00
Zanie Blue 572551c108
Refactor toolchain discovery to use `satisfies_system_python` explicitly (#4310)
Splitting out the refactor from
https://github.com/astral-sh/uv/pull/4309
2024-06-13 17:44:53 +00:00
Zanie Blue b07c132ede
Skip invalid interpreters when searching for requested interpreter executable name (#4308)
Previously, we took the first executable on the `PATH` but if it was not
a usable interpreter we'd fail. Now, we'll continue searching in the
path until we find an interpreter as we do with the standard executable
names.
2024-06-13 12:36:12 -05:00
konsti 30126950fe
Fix relative and absolute path handling in lockfiles (#4266)
Previously, `b` in the test case would have been incorrectly locked to
the path of `a`. I've moved `relative_to` into uv-fs since it's now used
in two different places.

Previously failing lockfile when `a/pyproject.toml` and
`a/b/pyproject.toml` exist (not in a workspace) and `a` was depending on
`b`:

```toml
version = 1
requires-python = ">=3.11, <3.13"

[[distribution]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
sdist = { path = "/home/konsti/projects/uv/a" }

[[distribution]]
name = "black"
version = "0.1.0"
source = "editable+."
sdist = { path = "." }

[[distribution.dependencies]]
name = "b"
version = "0.1.0"
source = "directory+/home/konsti/projects/uv/a"
```
2024-06-13 11:51:08 -04:00
Zanie Blue 89daa51dbe
Add support for listing system toolchains (#4172)
Includes system interpreters in `uv toolchain list`.

This includes a refactor of `find_toolchain` to support iterating over
all toolchains
that match a request rather than ending earlier.
2024-06-13 10:25:30 -05:00
Charlie Marsh b1bf7f0524
Ignore `compile_invalid_pyc_invalidation_mode` on macOS (#4305)
## Summary

This keeps failing in CI.
2024-06-13 10:08:28 -04:00
Brian Mego cd461f1243
Allow `--no-binary` with `uv pip compile` (#4301)
## Summary

This i still a draft, but it gets some of the work started on issue
#4064 , which requests --no-binary functionality similar to `uv pip
install` to exist on `uv pip compile`.

So far this has moved the command line shape from install and cloned it
over to compile. The actual functionality of respecting --no-binary
<package> and generating the resulting line in requirements.txt I could
use a hand with.

My understanding is we want to create a requirements.in file such as:

```
yt
```

Then when running `cargo run -- pip compile --no-binary yt
requirements.in`
we want the file to have this line in it:

```
yt==4.3.1 --no-binary yt
```

## Test Plan

Existing unit tests continue to pass.
No new unit tests have been created yet.
The new command line options do show up when testing with `cargo run --
pip compile -h`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-06-13 09:59:03 -04:00
Charlie Marsh 5d1305aa6b
Respect workspace-wide `requires-python` in interpreter selection (#4298)
## Summary

Closes https://github.com/astral-sh/uv/issues/4296.

## Test Plan

Ran `cargo run lock --verbose` from
`scripts/workspaces/albatross-virtual-workspace`:

```
DEBUG uv 0.2.11 (ef3bc1612 2024-06-12)
warning: `uv lock` is experimental and may change without warning.
DEBUG Found workspace root: `/Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace`
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/albatross
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
DEBUG Adding discovered workspace member: /Users/crmarsh/workspace/puffin/scripts/workspaces/albatross-virtual-workspace/packages/seeds
DEBUG Searching for Python >=3.12 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/crmarsh/Library/Application Support/uv/toolchains`
DEBUG Found managed toolchain `cpython-3.12.3-macos-aarch64-none`
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3` (managed toolchains)
Using Python 3.12.3 interpreter at: /Users/crmarsh/Library/Application Support/uv/toolchains/cpython-3.12.3-macos-aarch64-none/install/bin/python3
```
2024-06-13 12:55:56 +00:00
Zanie Blue b43de79275
Fix incorrect parsing of requested Python version as empty version specifiers (#4289)
Before 0.2.10 we would parse `--python=python` as an executable name.
After https://github.com/astral-sh/uv/pull/4214, we started treating
this as a Python version range request (with an empty version range).
This is not entirely unreasonable, but it was an unexpected regression
and I don't think `VersionRequest` should support empty ranges in its
`from_str` implementation without more consideration.
2024-06-12 19:48:59 -05:00
Charlie Marsh a547d7f9dc
Hide `--no-system` from the CLI (#4292) 2024-06-12 20:22:34 -04:00
Charlie Marsh 544b900432
Tweak copy on some command-line arguments (#4293) 2024-06-12 20:18:22 -04:00
Zanie Blue 5a007b6b9f
Add `BuildOptions` for centralized combination of `NoBuild` and `NoBinary` (#4284)
As requested in review of https://github.com/astral-sh/uv/pull/4067
2024-06-12 21:33:33 +00:00
Zanie Blue 1ab4041baa
Allow specific `--only-binary` and `--no-binary` packages to override `:all:` (#4067)
Updates `--no-binary <package>` to take precedence over `--only-binary
:all:` and `--only-binary <package>` to take precedence over
`--no-binary :all:`.

I'm not entirely sure about this behavior, e.g. maybe I provided
`--only-binary :all:` later on the command line and really want it to
override those earlier arguments of `--no-binary <package>` for safety.
Right now we just fail to solve though since we can't satisfy the
overlapping requests.

Closes https://github.com/astral-sh/uv/issues/4063
2024-06-12 15:47:45 -05:00
Charlie Marsh cf830288f3
Rename some internal settings structs (#4288)
## Summary

In preparation for future refactors.
2024-06-12 20:35:41 +00:00
Charlie Marsh d8f1de6134
Use separate path types for directories and files (#4285)
## Summary

This is what I consider to be the "real" fix for #8072. We now treat
directory and path URLs as separate `ParsedUrl` types and
`RequirementSource` types. This removes a lot of `.is_dir()` forking
within the `ParsedUrl::Path` arms and makes some states impossible
(e.g., you can't have a `.whl` path that is editable). It _also_ fixes
the `direct_url.json` for direct URLs that refer to files. Previously,
we wrote out to these as if they were installed as directories, which is
just wrong.
2024-06-12 15:59:21 -04:00
Charlie Marsh c4483017ac
Add `UV_EXCLUDE_NEWER` environment variable (#4287)
## Summary

Closes https://github.com/astral-sh/uv/issues/4286.
2024-06-12 15:54:01 -04:00
Zanie Blue 3910b7a90c
Respect `requires-python` in `uv lock` (#4282)
We weren't using the common interface in `uv lock` because it didn't
support finding an interpreter without touching the virtual environment.
Here I refactor the project interface to support what we need and update
`uv lock` to use the shared implementation.
2024-06-12 13:19:00 -05:00
Andrew Gallant e6d0c4d9fe uv/tests: add Requires-Python packse tests
Ref https://github.com/astral-sh/packse/pull/187
Ref https://github.com/astral-sh/packse/pull/188
Ref https://github.com/astral-sh/packse/pull/189
2024-06-12 13:30:47 -04:00
Andrew Gallant 75b323232d uv-resolver: use Requires-Python to filter dependencies during universal resolution
In the time before universal resolving, we would always pass a
`MarkerEnvironment`, and this environment would capture any relevant
`Requires-Python` specifier (including if `-p/--python` was provided on
the CLI).

But in universal resolution, we very specifically do not use a
`MarkerEnvironment` because we want to produce a resolution that is
compatible across potentially multiple environments. This in turn meant
that we lost `Requires-Python` filtering.

This PR adds it back. We do this by converting our `PythonRequirement`
into a `MarkerTree` that encodes the version specifiers in a
`Requires-Python` specifier. We then ask whether that `MarkerTree` is
disjoint with a dependency specification's `MarkerTree`. If it is, then
we know it's impossible for that dependency specification to every be
true, and we can completely ignore it.
2024-06-12 13:30:47 -04:00
Charlie Marsh c32667caec
Avoid treating direct path archives as always dynamic (#4283)
## Summary

Right now, we're _always_ reinstalling local wheel archives, even if the
timestamp didn't change.

I want to fix the TODO properly but I will do so in a separate PR.
2024-06-12 17:28:29 +00:00
Charlie Marsh baee826517
Use `FxHashMap` for available versions (#4278)
## Summary

I don't think we ever iterate over these in-order so I'd rather use
`FxHash` to avoid creating the appearance that the order matters.
2024-06-12 13:16:37 -04:00
Charlie Marsh 44f1afd6b0
Use registry URL for fetching source distributions from lockfile (#4280)
## Summary

This is just a logic bug with no testing. We were using the registry URL
(like `https://pypi.org/simple`) as the URL from which a source
distribution should be downloaded.

Closes https://github.com/astral-sh/uv/issues/4281.

## Test Plan

`cargo test`
2024-06-12 17:01:29 +00:00
Charlie Marsh 16b4a886a8
Use consistent order for extra groups in lockfile (#4275)
## Summary

Closes #4274.
2024-06-12 11:46:16 -04:00
Charlie Marsh aef74dac2c
Allow normalization to completely eliminate markers (#4271)
## Summary

`normalize` now takes an owned value and returns `Option<MarkerTree>`,
such that if any sub-expression evaluates to `true`, we can normalize
out the entire marker.

Closes https://github.com/astral-sh/uv/issues/4267.
2024-06-12 10:25:43 -04:00
Zanie Blue f7f55ede2f
Refactor `uv-toolchain::platform` to use `target-lexicon` (#4236)
Closes https://github.com/astral-sh/uv/issues/3857

Instead of using custom `Arch`, `Os`, and `Libc` types I just use
`target-lexicon`'s which enumerate way more variants and implement
display and parsing. We use a wrapper type to represent a couple special
cases to support the "x86" alias for "i686" and "macos" for "darwin".
Alternatively we could try to use our `platform-tags` types but those
capture more information (like operating system versions) that we don't
have for downloads.

As discussed in https://github.com/astral-sh/uv/pull/4160, this is not
sufficient for proper libc detection but that work is larger and will be
handled separately.
2024-06-12 09:11:56 -05:00
Nicolas Delaby 5a09c26e77
Fix doc for `uv add` cli command s/remove/add/ (#4269)
## Summary

Fix the docsting where `remove` was used instead of `add` in the context
of `uv add` command.

## Test Plan

```
cargo run -- add --help
```
```
Add one or more packages to the project requirements

Usage: uv add [OPTIONS] <REQUIREMENTS>...

Arguments:
  <REQUIREMENTS>...
          The packages to add, as PEP 508 requirements (e.g., `flask==2.2.3`)

```
2024-06-12 13:44:08 +00:00
Charlie Marsh 5e7b98d3e7
Use relative path for lib64 symlink (#4268)
## Summary

Closes https://github.com/astral-sh/uv/issues/4265.

## Test Plan

```
❯ ls -l .venv
total 16
-rw-r--r--   1 crmarsh  staff   43 Jun 12 09:23 CACHEDIR.TAG
drwxr-xr-x  14 crmarsh  staff  448 Jun 12 09:23 bin
drwxr-xr-x   3 crmarsh  staff   96 Jun 12 09:23 lib
lrwxr-xr-x   1 crmarsh  staff    3 Jun 12 09:23 lib64 -> lib
-rw-r--r--   1 crmarsh  staff  174 Jun 12 09:23 pyvenv.cfg
```

```
❯ ls .venv/lib64/
python3.12
```
2024-06-12 09:36:27 -04:00
Charlie Marsh 034b4790ea
Add uv version to debug output (#4259)
## Summary

I think this is a useful piece of connective tissue that will let us
avoid back-and-forths when folks include traces.

## Test Plan

```
❯ cargo run pip list --verbose
DEBUG uv 0.2.11 (44041bccd 2024-06-11)
DEBUG Searching for Python interpreter in virtual environments
DEBUG Found CPython 3.12.3 at `/Users/crmarsh/workspace/puffin/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
```
2024-06-11 22:59:31 -04:00
Charlie Marsh 8a8e1af513
Deduplicate markers during normalization (#4263)
## Summary

We need to sort _before_ deduplicating; otherwise, we can't detect
adjacent elements, so we aren't guaranteed to deduplicate anything.
2024-06-12 02:38:15 +00:00
Charlie Marsh cc2e9ec111
DRY up index argument parsing (#4262)
## Summary

Stealing this from @ibraheemdev's
https://github.com/astral-sh/uv/pull/4237 because it's a nice, isolated
change and I'm about to build atop it.
2024-06-11 22:25:28 -04:00
Charlie Marsh a1aa35b640
Move project commands into their own subcommand struct (#4261)
## Summary

No changes to the CLI itself (since this is flattened); just code
reorganization.
2024-06-12 02:18:16 +00:00
Charlie Marsh 22795f85bc
Flatten ORs and ANDs in marker construction (#4260)
## Summary

If we're ORing an OR, we should just append rather than nesting in
another OR.

In my branch, this let us simplify:

```
python_version < '3.10' or python_version > '3.12' or (python_version < '3.8' or python_version > '3.12')
```

To:

```
python_version < '3.10' or python_version > '3.12
```
2024-06-11 21:44:49 -04:00
Charlie Marsh 44041bccd2
Bump version to v0.2.11 (#4258) 2024-06-11 20:47:25 -04:00
Charlie Marsh 44592681a0
Represent build tag as `u64` (#4253)
## Summary

The build tags in this case are like, e.g., `202206090410`. That's
larger than a `u32`, so we're rejecting the wheel. In theory build tags
could be even larger, but we already use `u64` for version segment so I
think it's fine to keep that constraint here.

I'm going to look into surfacing these errors separately.

Closes https://github.com/astral-sh/uv/issues/4252.

## Test Plan

`cargo run pip install monailabel`
2024-06-11 21:40:08 +00:00
Charlie Marsh 6dae1920af
Make missing `METADATA` file a recoverable error (#4247)
## Summary

I don't have a great way to test it, but this makes the error described
in https://github.com/astral-sh/uv/issues/4246 an incompatibility rather
than a fatal error.

Closes https://github.com/astral-sh/uv/issues/4246.
2024-06-11 19:49:38 +00:00
Zanie Blue 8cfe202e4e
Improve output when an older toolchain version is already installed (#4248)
e.g.

```
❯ uv toolchain install
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
A toolchain is already installed. Use `uv toolchain install <request>` to install a specific toolchain
```

instead of

```
❯ uv toolchain install
Using latest Python version
Found installed toolchain 'cpython-3.9.19-macos-aarch64-none'
Already installed at /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.19-macos-aarch64-none
```
2024-06-11 19:49:15 +00:00
Charlie Marsh dce913c542
Warn when 'requires-python' does not include a lower bound (#4234)
## Summary

Closes https://github.com/astral-sh/uv/issues/4089.
2024-06-11 18:50:05 +00:00
Charlie Marsh b3a99d9ff9
Allow direct URLs for dev dependencies (#4233)
## Summary

Ensures that they're included in the lookahead resolver.

Closes https://github.com/astral-sh/uv/issues/4230.
2024-06-11 15:30:34 +00:00
Ibraheem Ahmed 3ea5a2a65d
De-duplicate resolution environment logic (#4232)
## Summary

Cleans up some duplicated logic in `pip install/sync/compile`. Will also
be needed for `uv add`.
2024-06-11 11:13:30 -04:00
Ibraheem Ahmed a75ed22a2f
Fix outdated comment (#4229)
## Summary

Looks like this wasn't updated at some point. I think it's referring to
`UnresolvedRequirement`?
2024-06-11 16:06:47 +02:00
Ibraheem Ahmed eefa9e62fc
Initial implementation of `uv add` and `uv remove` (#4193)
## Summary

Basic implementation of `uv add` and `uv remove` that supports writing
PEP508 requirements to `project.dependencies`.

First step for https://github.com/astral-sh/uv/issues/3959 and
https://github.com/astral-sh/uv/issues/3960.
2024-06-11 09:21:28 -04:00
Nyakku Shigure 60431ce78c
Replace `toolchain fetch` with `toolchain install` (#4228)
## Summary

Something that looks like it was forgotten to replace in #4164.

## Test Plan

Run `cargo run toolchain install` should display the warning: ``warning:
`uv toolchain install` is experimental and may change without warning.``
2024-06-11 08:00:33 -04:00
konsti 44833801b3
Support locking relative paths (#4205)
By splitting `path` into a lockable, relative (or absolute) and an
absolute installable path and by splitting between urls and paths by
dist type, we can store relative paths in the lockfile.
2024-06-11 11:58:03 +00:00
Charlie Marsh 33cf47182f
Migrate lock errors to `thiserror` (#4225)
## Summary

Do we prefer this?
2024-06-11 07:40:48 -04:00
Charlie Marsh 656fc427b9
Add support for local directories with `--index-url` (#4226)
## Summary

Closes #4078.
2024-06-10 22:27:04 -04:00
Zanie Blue f296ef08d6
Refactor project interpreter request for `requires-python` specifiers (#4216)
Refactor following #4214 to avoid parsing the specifiers again
2024-06-10 20:32:40 -05:00
samypr100 68abf85f0d
feat: mTLS support (#4171)
## Summary

Closes https://github.com/astral-sh/uv/issues/3626

This adds mTLS support to uv via the standard env var `SSL_CLIENT_CERT`.

## Test Plan

Tested locally using a [nginx proxy to
pypi](https://github.com/hauntsaninja/nginx_pypi_cache) using my own
self-signed ca + certs + client certs generated via
[mkcert](https://github.com/FiloSottile/mkcert). Used this proxy with
both uv and pip to make sure we have feature partity in mTLS
functionality.
2024-06-10 20:11:35 -05:00
Zanie Blue 5f37395f45
Allow version specifiers to be used in Python version requests (#4214)
In service of https://github.com/astral-sh/uv/issues/4212 but this is
user-facing e.g. Python discovery will support version specifiers
everywhere now.

Closes https://github.com/astral-sh/uv/issues/4212
2024-06-10 18:20:09 -05:00
Charlie Marsh 10e0abc9b1
Create temporary environments in dedicated cache bucket (#4223)
## Summary

We may choose to persist these eventually, but for now, it's useful to
have them colocated with the cache, and in their own dedicated bucket
(so, at the very least, we can keep track of the use-cases).

Closes https://github.com/astral-sh/uv/issues/4219.
2024-06-10 23:17:35 +00:00
Zanie Blue b5c9003883
Fix `pip_compile::missing_venv` test (#4224)
A merge kerfuffle from #4222  and #4218 

Now we fail because we genuinely can't find any interpreters since tests
contexts are isolated by default. I'll improve the error message and
maybe add another test case once `main` is fixed.
2024-06-10 22:39:25 +00:00
Zanie Blue 546e23bb1d
Update test context to avoid discovery of external Pythons (#4222)
By setting the test search path to an empty path, we avoid accidentally
pulling interpreters from the system during a test case.

Cherry-picked from https://github.com/astral-sh/uv/pull/4214
2024-06-10 17:26:53 -05:00
Zanie Blue 98d1ea6bb0
Improve handling of missing interpreters during discovery (#4218)
Cherry-picked from https://github.com/astral-sh/uv/pull/4214

The first commit gets us some context on an IO error during queries:

Previously:

```
failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

Now:

```
Failed to query Python interpreter
    Caused by: failed to canonicalize path `[VENV]/bin/python3`
    Caused by: No such file or directory (os error 2)
```

but really we shouldn't attempt to query a missing interpreter during
discovery anyway, so we improve handling of that too.
2024-06-10 22:26:34 +00:00
Charlie Marsh 0c1dcb797a
Remove usages of verbatim URL in URL resolver (#4221)
## Summary

Should be no behavior changes, but one piece of technical debt I noticed
left over in the URL resolver. We already have structured paths, so we
shouldn't need to compare verbatim URLs.
2024-06-10 21:55:48 +00:00
Zanie Blue fd52fe74ce
Update the `Toolchain::find_requested` API to take a parsed request (#4215)
Pulled out of https://github.com/astral-sh/uv/pull/4206, need this for
#4214
2024-06-10 20:07:45 +00:00
Charlie Marsh d7e10bb4a2
Remove project dependency from `sync` (#4211)
## Summary

This is architecturally desirable because it means syncing is
independent from the project (e.g., the `pyproject.toml` and friends).
2024-06-10 19:23:37 +00:00
Charlie Marsh e31604e38b
Always install as editable when duplicates are requested (#4208)
## Summary

If the user requests a package as both editable and non-editable, the
editable now "wins".

Previously, `pip install -e . .` would install as editable. However,
`pip install -e . -r requirements.txt` would _not_ if `requirements.txt`
contained `.`, because we ignored `editable` when deduplicating and the
order of iteration was just dependent on internals.

Closes https://github.com/astral-sh/uv/issues/4053.
2024-06-10 15:02:17 -04:00
Zanie Blue 06a0fc65d4
Improve `uv toolchain list` implementation (#4203)
Amends #4163 with review from Jane, thank you!

No behavior changes.
2024-06-10 17:49:17 +00:00
Charlie Marsh 652c1126d3
Avoid crash for `XDG_CONFIG_HOME=/dev/null` (#4200)
## Summary

Closes https://github.com/astral-sh/uv/issues/4199.

## Test Plan

`XDG_CONFIG_HOME=/dev/null cargo run venv`
2024-06-10 17:36:29 +00:00
Zanie Blue 907727cb1b
Bump version to 0.2.10 (#4201) 2024-06-10 11:40:19 -05:00
Zanie Blue 14d535f384
Add `uv toolchain install` (#4164)
Adds a command (following #4163) to download and install specific
toolchains. While we fetch toolchains on demand, this is useful for,
e.g., pre-downloading a toolchain in a Docker image build.

~I kind of think we should call this `install` instead of `fetch`~ I
changed the name from `fetch` to `install`.
2024-06-10 10:56:08 -05:00
konsti 90947a933d
Recreate project environment if `--python` or `requires-python` doesn't match (#3945)
Fixes #4131
Fixes #3895
2024-06-10 14:36:14 +00:00
Charlie Marsh 125a4b220e
Improve static metadata extraction for Poetry projects (#4182)
## Summary

Adds handling for a few cases to improve interoperability with Poetry:

- If the `project` schema is invalid, we now raise a hard error, rather
than treating the metadata as dynamic and then falling back to the build
backend. This could cause problems, I'm not sure. It's stricter than
before.
- If the project contains `tool.poetry` but omits
`project.dependencies`, we now treat it as dynamic. We could go even
further and treat _any_ Poetry project as dynamic, but then we'd be
ignoring user-declared dependencies, which is also confusing.

Closes https://github.com/astral-sh/uv/issues/4142.
2024-06-10 10:26:38 -04:00
Zanie Blue c6da4f15b7
Add `uv toolchain list` (#4163)
Adds the `uv toolchain` namespace and a `list` command to get us
started.

```
❯ cargo run -q -- toolchain list
warning: `uv toolchain list` is experimental and may change without warning.
3.8.12   (cpython-3.8.12-macos-aarch64-none)
3.8.13   (cpython-3.8.13-macos-aarch64-none)
3.8.14   (cpython-3.8.14-macos-aarch64-none)
3.8.15   (cpython-3.8.15-macos-aarch64-none)
3.8.16   (cpython-3.8.16-macos-aarch64-none)
3.8.17   (cpython-3.8.17-macos-aarch64-none)
3.8.18   (cpython-3.8.18-macos-aarch64-none)
3.8.18   (cpython-3.8.18-macos-aarch64-none)
3.8.19   (cpython-3.8.19-macos-aarch64-none)
3.9.2    (cpython-3.9.2-macos-aarch64-none)
3.9.3    (cpython-3.9.3-macos-aarch64-none)
3.9.4    (cpython-3.9.4-macos-aarch64-none)
3.9.5    (cpython-3.9.5-macos-aarch64-none)
3.9.6    (cpython-3.9.6-macos-aarch64-none)
3.9.7    (cpython-3.9.7-macos-aarch64-none)
3.9.10   (cpython-3.9.10-macos-aarch64-none)
3.9.11   (cpython-3.9.11-macos-aarch64-none)
3.9.12   (cpython-3.9.12-macos-aarch64-none)
3.9.13   (cpython-3.9.13-macos-aarch64-none)
3.9.14   (cpython-3.9.14-macos-aarch64-none)
3.9.15   (cpython-3.9.15-macos-aarch64-none)
3.9.16   (cpython-3.9.16-macos-aarch64-none)
3.9.17   (cpython-3.9.17-macos-aarch64-none)
3.9.18   (cpython-3.9.18-macos-aarch64-none)
3.9.19   (cpython-3.9.19-macos-aarch64-none)
3.10.0   (cpython-3.10.0-macos-aarch64-none)
3.10.2   (cpython-3.10.2-macos-aarch64-none)
3.10.3   (cpython-3.10.3-macos-aarch64-none)
3.10.4   (cpython-3.10.4-macos-aarch64-none)
3.10.5   (cpython-3.10.5-macos-aarch64-none)
3.10.6   (cpython-3.10.6-macos-aarch64-none)
3.10.7   (cpython-3.10.7-macos-aarch64-none)
3.10.8   (cpython-3.10.8-macos-aarch64-none)
3.10.9   (cpython-3.10.9-macos-aarch64-none)
3.10.11  (cpython-3.10.11-macos-aarch64-none)
3.10.12  (cpython-3.10.12-macos-aarch64-none)
3.10.13  (cpython-3.10.13-macos-aarch64-none)
3.10.14  (cpython-3.10.14-macos-aarch64-none)
3.11.1   (cpython-3.11.1-macos-aarch64-none)
3.11.3   (cpython-3.11.3-macos-aarch64-none)
3.11.4   (cpython-3.11.4-macos-aarch64-none)
3.11.5   (cpython-3.11.5-macos-aarch64-none)
3.11.6   (cpython-3.11.6-macos-aarch64-none)
3.11.7   (cpython-3.11.7-macos-aarch64-none)
3.11.8   (cpython-3.11.8-macos-aarch64-none)
3.11.9   (cpython-3.11.9-macos-aarch64-none)
3.12.0   (cpython-3.12.0-macos-aarch64-none)
3.12.1   (cpython-3.12.1-macos-aarch64-none)
3.12.2   (cpython-3.12.2-macos-aarch64-none)
3.12.3   (cpython-3.12.3-macos-aarch64-none)
```

Closes https://github.com/astral-sh/uv/issues/4189
2024-06-10 09:22:00 -05:00
Zanie Blue a2e6aaa0ff
Fetch managed toolchains in `uv run` (#4143) 2024-06-10 14:20:44 +00:00
Zanie Blue 45df889fe4
Implement `Toolchain::find_or_fetch` and use in `uv venv --preview` (#4138)
Extends https://github.com/astral-sh/uv/pull/4121
Part of #2607 

Adds support for managed toolchain fetching to `uv venv`, e.g.

```
❯ cargo run -q -- venv --python 3.9.18 --preview -v
DEBUG Searching for Python 3.9.18 in search path or managed toolchains
DEBUG Searching for managed toolchains at `/Users/zb/Library/Application Support/uv/toolchains`
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Found CPython 3.9.6 at `/usr/bin/python3` (search path)
DEBUG Found CPython 3.12.3 at `/opt/homebrew/bin/python3` (search path)
DEBUG Requested Python not found, checking for available download...
DEBUG Using registry request timeout of 30s
INFO Fetching requested toolchain...
DEBUG Downloading https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst to temporary location /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp
DEBUG Extracting cpython-3.9.18%2B20240224-aarch64-apple-darwin-pgo%2Blto-full.tar.zst
DEBUG Moving /Users/zb/Library/Application Support/uv/toolchains/.tmpgohKwp/python to /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none
Using Python 3.9.18 interpreter at: /Users/zb/Library/Application Support/uv/toolchains/cpython-3.9.18-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
INFO Removing existing directory
Activate with: source .venv/bin/activate
```

The preview flag is required. The fetch is performed if we can't find an
interpreter that satisfies the request. Once fetched, the toolchain will
be available for later invocations that include the `--preview` flag.
There will be follow-ups to improve toolchain management in general,
there is still outstanding work from the initial implementation.
2024-06-10 14:10:45 +00:00
Charlie Marsh 04c4da4e65
Fix `>=` to `==` typo in `is_contained_by` docs (#4196) 2024-06-10 13:28:33 +00:00
Charlie Marsh 72bc739a64
Remove `PubGrubPackage` dependency from `ResolutionGraph` (#4168)
## Summary

Similar to how we abstracted the dependencies into
`ResolutionDependencyNames`, I think it makes sense to abstract the base
packages into a `ResolutionPackage`. This also avoids leaking details
about the various `PubGrubPackage` enum variants to `ResolutionGraph`.
2024-06-10 12:50:32 +00:00
konsti 18b40b0c7d
Don't panic with invalid wheel source (#4191)
Remove the panic when there is an invalid wheel source, instead surface
the error. This error can only occur when manually editing the lock
file, but since it's an external file, we should error and not panic.

This change is helpful since the method needs to be able to error for
relative path support.
2024-06-10 08:44:36 -04:00
Charlie Marsh 763e2d2e84
Add markers to edges rather than distributions (#4166)
## Summary

We've debated this a bit but the thing that tipped me over the edge is
https://github.com/astral-sh/uv/issues/4157. As-is, there's no way to
represent "a package should be installed, but the extra should only be
installed conditionally based on the markers", because the markers sit
on the _distribution_. By placing the markers on the edge, we can now
represent scenarios that weren't previously representable.

Closes https://github.com/astral-sh/uv/issues/4137.
Closes https://github.com/astral-sh/uv/issues/4125.
Closes https://github.com/astral-sh/uv/issues/4157.
2024-06-10 08:40:51 -04:00
Charlie Marsh 5269a0dba8
Ignore tags in universal resolution (#4174)
## Summary

If a package lacks a source distribution, and we can't find a compatible
wheel for the current platform, we need to just _assume_ that the
package will have a valid wheel on all platforms on which it's
requested; if not, we raise an error at install time.

It's possible that we can be smarter about this over time. For example,
if the package was requested _only_ for macOS, we could verify that
there's at least one macOS-compatible wheel. See the linked issue for
more details.

Closes https://github.com/astral-sh/uv/issues/4139.
2024-06-10 08:38:21 -04:00
Charlie Marsh 4125cb26b8
Add `pyproject.toml` to CLI help (#4181) 2024-06-09 22:48:39 -04:00
Charlie Marsh e7c573cfcb
Fix existing typos and enable `typos` in CI (#4184) 2024-06-10 01:50:54 +00:00
Charlie Marsh 04717e1a92
Respect `--find-links` in `lock` and `sync` (#4183)
## Summary

We already respect this command-line argument, I just forgot to wire it
up to the resolver.
2024-06-09 21:42:52 -04:00
Charlie Marsh 72859d8f9b
Move PEP 440 and PEP 508 parsing out of TOML deserialization (#4176)
## Summary

If we want more granular control over how these errors are handled, then
we need to move them out of the TOML deserialization.

No actual behavior changes here.

Part of https://github.com/astral-sh/uv/issues/4142.
2024-06-10 00:09:39 +00:00
Charlie Marsh a761047df9
Reduce visibility of `pyproject.toml` fields (#4175) 2024-06-09 23:57:43 +00:00
Charlie Marsh 8ae5c2aee3
Skip version iteration for non-base packages (#4167) 2024-06-08 18:46:08 +00:00
Charlie Marsh ab0f8afe1f
Remove version from graph edge (#4165)
## Summary

We're actually not using the edge data, so lets remove it.
2024-06-08 17:39:02 +00:00
Zanie Blue 7c07ca8090
Fix test snapshot for `lock_conditional_dependency_extra` (#4159)
Introduced in #4156 and is failing on main
2024-06-08 12:04:44 -05:00
Charlie Marsh 477c860c9b
Add incorrect conditional-extra test (#4156) 2024-06-07 21:24:35 -04:00
Charlie Marsh eb239ff640
Cap `Requires-Python` comparisons at the patch version (#4150)
## Summary

See the long comment inline. I think this is debatable but probably
right for now. The other options have their own problems, but there are
a few alternate ideas in the comment.

Closes https://github.com/astral-sh/uv/issues/4132.
2024-06-08 01:22:57 +00:00
Charlie Marsh ac1ddf5e4e
Allow transitive URLs via recursive extras (#4155)
## Summary

Closes https://github.com/astral-sh/uv/issues/4152.
2024-06-08 01:10:18 +00:00
Andrew Gallant c46fa74e65
make universal resolver fork only when markers are disjoint (#4135)
The basic idea here is to make it so forking can only ever result in a
resolution that, for a particular marker environment, will only install
at most one version of a package. We can guarantee this by ensuring we
only fork on conflicting dependency specifications only when their
corresponding markers are completely disjoint. If they aren't, then
resolution _must_ find a single version of the package in the
intersection of the two dependency specifications.

A test for this case has been added to packse here:
https://github.com/astral-sh/packse/pull/182. Previously, that test
would result in a resolution with two different unconditional versions
of the same package. With this change, resolution fails (as it should).

A commit-by-commit review should be helpful here, since the first commit
is a refactor to make the second commit a bit more digestible.
2024-06-07 23:40:55 +00:00
Charlie Marsh 0db1bf4df7
Avoid pre-fetching for unbounded minimum versions (#4149)
## Summary

I think we should be able to model PubGrub such that this isn't
necessary (at least for the case described in the issue), but for now,
let's just avoid attempting to build very old distributions in
prefetching.

Closes https://github.com/astral-sh/uv/issues/4136.
2024-06-07 22:05:14 +00:00
Charlie Marsh d7cc622d6c
Accept `file://` URLs for `requirements.txt` et all references (#4145)
## Summary

Closes https://github.com/astral-sh/uv/issues/4124.
2024-06-07 22:03:08 +00:00
Zanie Blue e3b274413d
Do not create a virtual environment when locking (#4147)
Closes https://github.com/astral-sh/uv/issues/4141
2024-06-07 16:56:45 -05:00
Zanie Blue f9ea304be4
Drop "registry" prefix from request timeout log (#4144)
We use this base client for more than registry requests
2024-06-07 16:56:32 -05:00
Zanie Blue 13f03e9d3f
Reduce `uv-toolchain` discovery API to `Toolchain` (#4148)
Drops `find_toolchain`, `find_best_toolchain`, etc. in favor of
`Toolchain::find_...`

We can change this in the future, but there should only be one "right"
way to do it not two redundant ways in the public interface.
2024-06-07 16:56:25 -05:00
Zanie Blue 365bcfce1a
Update `init_environment` to use `Toolchain::find_default` API (#4146)
In service of https://github.com/astral-sh/uv/pull/4148
2024-06-07 16:16:58 -05:00
Ibraheem Ahmed 7232c53718
Simplify marker expressions in lockfile (#4066)
## Summary

Simplify and normalize marker expressions in the lockfile. Right now
this does a simple analysis by only looking at related operators at the
same level of precedence. I think anything more complex would be out of
scope.

Resolves https://github.com/astral-sh/uv/issues/4002.
2024-06-07 16:14:24 -04:00
Charlie Marsh bcfe88dfdc
Track `Markers` via a PubGrub package variant (#4123)
## Summary

This PR adds a lowering similar to that seen in
https://github.com/astral-sh/uv/pull/3100, but this time, for markers.
Like `PubGrubPackageInner::Extra`, we now have
`PubGrubPackageInner::Marker`. The dependencies of the `Marker` are
`PubGrubPackageInner::Package` with and without the marker.

As an example of why this is useful: assume we have `urllib3>=1.22.0` as
a direct dependency. Later, we see `urllib3 ; python_version > '3.7'` as
a transitive dependency. As-is, we might (for some reason) pick a very
old version of `urllib3` to satisfy `urllib3 ; python_version > '3.7'`,
then attempt to fetch its dependencies, which could even involve
building a very old version of `urllib3 ; python_version > '3.7'`. Once
we fetch the dependencies, we would see that `urllib3` at the same
version is _also_ a dependency (because we tack it on). In the new
scheme though, as soon as we "choose" the very old version of `urllib3 ;
python_version > '3.7'`, we'd then see that `urllib3` (the base package)
is also a dependency; so we see a conflict before we even fetch the
dependencies of the old variant.

With this, I can successfully resolve the case in #4099.

Closes https://github.com/astral-sh/uv/issues/4099.
2024-06-07 19:57:02 +00:00
Zanie Blue 0f4f3b4714
Update `uv run` and `uv tool run` to use `Toolchain::find` (#4134)
Extends https://github.com/astral-sh/uv/pull/4121
2024-06-07 19:28:59 +00:00
Zanie Blue 53035d65a1
Refactor `uv-toolchain` types (#4121)
Extends #4120 
Part of #2607 

There should be no behavior changes here. Restructures the discovery API
to be focused on a toolchain first perspective in preparation for
exposing a `find_or_fetch` method for toolchains in
https://github.com/astral-sh/uv/pull/4138.
2024-06-07 14:20:28 -05:00
Zanie Blue 325982c418
Rename `uv-interpreter` crate to `uv-toolchain` (#4120)
In preparation for managed toolchains #2607, just renames the crate to
something broader.

See #4121 and https://github.com/astral-sh/uv/pull/4138 to see the final
intent.
2024-06-07 13:59:14 -05:00
Charlie Marsh 2803a8c475
Omit URL dependencies from pre-release hints (#4140)
## Summary

Closes https://github.com/astral-sh/uv/issues/4127.
2024-06-07 18:43:50 +00:00
Charlie Marsh 7d1b7b99d9
Rename `Dependency.id` to `Dependency.distribution_id` (#4114)
## Summary

I think this makes clearer that the `Dependency.id` is not an identifier
for the dependency itself.

No functional changes.
2024-06-07 18:28:54 +00:00
Charlie Marsh fa10679275
Add extra and dev dependency validation to lockfile (#4112)
## Summary

Closes https://github.com/astral-sh/uv/issues/4106.

Closes https://github.com/astral-sh/uv/issues/4115.
2024-06-07 14:18:31 -04:00
Charlie Marsh d3651c13f0
Set `--dev` to default for `uv run` and `uv sync` (#4118)
## Summary

Closes https://github.com/astral-sh/uv/issues/4117.
2024-06-07 00:06:05 +00:00
Zanie Blue dcf70a1f29
Include non-standard ports in keyring host queries (#4061)
Partially addresses https://github.com/astral-sh/uv/issues/4056

We were incorrectly omitting the port from requests to `keyring` when
falling back to a realm/host query, e.g. `localhost` was used instead of
`localhost:1234`. We still won't include "standard" ports like `80` for
an HTTP request.
2024-06-06 19:02:47 -05:00
Zanie Blue cb7d6245ae
Use `PythonEnvironment` API in `uv pip compile` (#4030)
Same as https://github.com/astral-sh/uv/pull/4029
2024-06-06 18:54:40 -05:00
Charlie Marsh cc7c780523
Remove PubGrub dependency from `uv` (#4116)
## Summary

Encapsulates more of the details are `Requires-Python` and PubGrub.

Closes https://github.com/astral-sh/uv/issues/4110.
2024-06-06 23:45:58 +00:00
Charlie Marsh 52bdee2e85
Add support for `--prefix` (#4085)
## Summary

Closes #3076.
2024-06-06 16:15:28 -04:00
Charlie Marsh 677a7f157b
Avoid showing dev hints for Python requirements (#4111)
Closes https://github.com/astral-sh/uv/issues/4096.
2024-06-06 19:58:30 +00:00
konsti e4e2590076
Use union of `requires-python` in workspace (#4041)
Follow-up to #4016.

This exposes `Range` and `PubGrubSpecifier` from outside the resolver to
use pubgrub's union creating a dependency edge we don't really want.
2024-06-06 19:21:02 +00:00
konsti a6f53e2aa4
Lock all packages in workspace (#4016)
When creating a lockfile, lock the combined dependencies for all
packages in a workspace. This make the lockfile independent of where you
are in the workspace.

Fixes #3983
2024-06-06 19:09:44 +00:00
Charlie Marsh e9fc99e622
Bump version to v0.2.9 (#4107) 2024-06-06 14:21:33 -04:00
Charlie Marsh 31bb01f0be
Ignore upper-bounds on `Requires-Python` (#4086)
## Summary

This PR modifies our `Requires-Python` handling to treat
`Requires-Python` as a lower bound. There's extensive discussion around
this in https://github.com/astral-sh/uv/issues/4022 and the references
linked therein. I think it's an experiment worth trying. Even in my own
small projects, I'm running into issues whereby I'm being "forced" to
add a `<4` upper bound to my `Requires-Python` due to these caps.

Separately, we should explore adding a mechanism that's distinct from
`Requires-Python` to enable users to declare a supported range for
locking.

Closes https://github.com/astral-sh/uv/issues/4022.
2024-06-06 13:52:57 -04:00
Charlie Marsh 30e73a60de
Avoid enforcing distribution ID uniqueness for extras (#4104)
## Summary

The condition enforced here isn't quite right. The same dependency can
appear multiple times, as long as the extra is different.

Closes https://github.com/astral-sh/uv/issues/4101.
2024-06-06 15:21:31 +00:00
Charlie Marsh 8798e91dd5
Avoid extra-only filtering for constraints (#4095)
## Summary

The "only include if relevant for the extra" filtering should _not_ be
applied to constraints. Otherwise, we'd only constrain when the extra
was included in the constraints file itself, which is incorrect.

Closes https://github.com/astral-sh/uv/issues/4091.
2024-06-06 13:58:46 +00:00
konsti 39f8978920
Fix logging in uv-dev (#4093)
Set the correct default logging directive and filtering in uv-dev. See
#4090.
2024-06-06 13:06:57 +00:00
Charlie Marsh fa2b6a28bc
Prefix sys fields with `sys_` consistently on Interpreter (#4084)
## Summary

This is more consistent across the struct _and_ opens up space for
`--prefix`.
2024-06-06 02:11:21 +00:00
Charlie Marsh 120148f0a1
Remove integration tests from `uv-resolver` (#4083)
## Summary

I don't think it's worth maintaining this separate test harness for ~18
tests, when they can all be tested in the `uv` package itself. Let's
drop the maintenance burden.
2024-06-06 01:48:42 +00:00
Charlie Marsh 0acae9bd9c
Add support for development dependencies (#4036)
## Summary

Externally, development dependencies are currently structured as a flat
list of PEP 580-compatible requirements:

```toml
[tool.uv]
dev-dependencies = ["werkzeug"]
```

When locking, we lock all development dependencies; when syncing, users
can provide `--dev`.

Internally, though, we model them as dependency groups, similar to
Poetry, PDM, and [PEP 735](https://peps.python.org/pep-0735). This
enables us to change out the user-facing frontend without changing the
internal implementation, once we've decided how these should be exposed
to users.

A few important decisions encoded in the implementation (which we can
change later):

1. Groups are enabled globally, for all dependencies. This differs from
extras, which are enabled on a per-requirement basis. Note, however,
that we'll only discover groups for uv-enabled packages anyway.
2. Installing a group requires installing the base package. We rely on
this in PubGrub to ensure that we resolve to the same version (even
though we only expect groups to come from workspace dependencies anyway,
which are unique). But anyway, that's encoded in the resolver right now,
just as it is for extras.
2024-06-06 01:40:17 +00:00
Charlie Marsh a81fb92ee6
Respect existing `.egg-link` files in site packages (#4082)
## Summary

As with other `.egg-info` and `.egg-link` distributions, it's easy to
support _existing_ `.egg-link` files. Like pip, we refuse to uninstall
these, since there's no way to know which files are part of the
distribution.

Closes https://github.com/astral-sh/uv/issues/4059.

## Test Plan

Verify that `vtk` is included here, which is installed as a `.egg-link`
file:

```
> conda create -c conda-forge -n uv-test python h5py vtk pyside6 cftime psutil

> cargo run pip freeze --python /opt/homebrew/Caskroom/miniforge/base/envs/uv-test/bin/python
aiohttp @ file:///Users/runner/miniforge3/conda-bld/aiohttp_1713964997382/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
cftime @ file:///Users/runner/miniforge3/conda-bld/cftime_1715919201099/work
frozenlist @ file:///Users/runner/miniforge3/conda-bld/frozenlist_1702645558715/work
h5py @ file:///Users/runner/miniforge3/conda-bld/h5py_1715968397721/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1713279365350/work
loguru @ file:///Users/runner/miniforge3/conda-bld/loguru_1695547410953/work
msgpack @ file:///Users/runner/miniforge3/conda-bld/msgpack-python_1715670632250/work
multidict @ file:///Users/runner/miniforge3/conda-bld/multidict_1707040780513/work
numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1707225421156/work/dist/numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl
pip==24.0
psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1705722460205/work
pyside6==6.7.1
setuptools==70.0.0
shiboken6==6.7.1
vtk==9.2.6
wheel==0.43.0
wslink @ file:///home/conda/feedstock_root/build_artifacts/wslink_1716591560747/work
yarl @ file:///Users/runner/miniforge3/conda-bld/yarl_1705508643525/work
```
2024-06-06 01:00:00 +00:00
Zanie Blue e6817bd997
Add regression test for `VIRTUAL_ENV` and `uv venv` interaction (#4074)
For https://github.com/astral-sh/uv/pull/4073

Demonstrated to fail without those changes.
2024-06-06 00:28:45 +00:00
Zanie Blue e48b9cd94b
Bump version to 0.2.8 (#4076) 2024-06-05 21:09:05 +00:00
Zanie Blue bacde427f1
Fix `uv venv` handling when `VIRTUAL_ENV` refers to an non-existant environment (#4073)
Closes https://github.com/astral-sh/uv/issues/4072

This was an accidental change in
https://github.com/astral-sh/uv/pull/4029 in which I had updated the
pull request to support virtual environments as requested in review and
forgot to revert it.

Separately, we shouldn't fail if `VIRTUAL_ENV` points to an empty
directory and `SystemPython::Allowed` is used, will address that
separately.
2024-06-05 20:50:28 +00:00
Charlie Marsh e5f95186de
Default to current Python minor if `Requires-Python` is absent (#4070)
## Summary

If `Requires-Python` is omitted in `uv lock` or `uv run`, we now warn
and default to `>=` the current minor version.

Closes https://github.com/astral-sh/uv/issues/4050.
2024-06-05 20:45:50 +00:00
Charlie Marsh 0b6d5b37dc
Enforce `Requires-Python` when syncing (#4068)
## Summary

Ensures that we raise if the user attempts to use a Python version that
wasn't included in the locked range.

Closes https://github.com/astral-sh/uv/issues/4052.
2024-06-05 16:22:08 -04:00
Charlie Marsh 642cef0dad
Track supported Python range in lockfile (#4065)
## Summary

This PR adds the `Requires-Python` range to the user's lockfile. This
will enable us to validate it when installing.

For now, we repeat the `Requires-Python` back to the user;
alternatively, though, we could detect the supported Python range
automatically.

See: https://github.com/astral-sh/uv/issues/4052
2024-06-05 16:21:59 -04:00
konsti 8596525d97
Update pubgrub to new `add_incompatibility_from_dependencies` (#4062)
We had previously changed the signature of
`DependencyProvider::get_dependencies` to return an iterator instead of
a hashmap to avoid the conversion cost from our dependencies `Vec` to
the pubgrub's hashmap. These changes are difficult to make in pubgrub
since they complicate the public api. But we don't actually use
`DependencyProvider::get_dependencies`, so we rolled those
customizations back in https://github.com/pubgrub-rs/pubgrub/pull/226
and instead opted to change only the internal
`add_incompatibility_from_dependencies` method that we exposed in our
fork. This aligns us closer with upstream, removes the design questions
about `DependencyProvider` from our concerns and reduces our diff (not
counting the github action) to +36 -12.
2024-06-05 20:46:00 +02:00
konsti 9f1b592fca
Retry flaky `compile_invalid_pyc_invalidation_mode` test (#4043)
Retry the flaky `compile_invalid_pyc_invalidation_mode` if it fails. I
don't understand why this happening in the first place (we have code
that should catch those cases, but also those cases shouldn't be
happening at all) and this is terrible hack, but it fixes the test
flakes.

Fixes #2672
2024-06-05 20:37:17 +02:00
Charlie Marsh a241f148de
Bump version to v0.2.7 (#4060) 2024-06-05 14:25:25 -04:00
Charlie Marsh 191f9556b7
Avoid building packages with dynamic versions (#4058)
## Summary

This PR separates "gathering the requirements" from the rest of the
metadata (e.g., version), which isn't required when installing a
package's _dependencies_ (as opposed to installing the package itself).
It thus ensures that we don't need to build a package when a static
`pyproject.toml` is provided in `pip compile`.

Closes https://github.com/astral-sh/uv/issues/4040.
2024-06-05 18:11:58 +00:00
Charlie Marsh a0173760f1
Remove unused PEP 621 error (#4057) 2024-06-05 17:45:42 +00:00
Charlie Marsh c97427d530
Reduce visibility of `lowering` (#4055)
## Summary

This makes `lowering.rs` internal to the metadata package.
2024-06-05 13:39:37 -04:00
Charlie Marsh 34f847bb68
Remove unused `Requirements` struct (#4054) 2024-06-05 17:14:57 +00:00
Zanie Blue db3c36dbcc
Use `PythonEnvironment` API in `uv venv` (#4029)
There's no reason to be reaching into the lower-level `find_interpreter`
manually here.
2024-06-05 11:49:29 -05:00
konsti b05a39c735
Store explicit project on workspace member (#4048)
We know that `[project]` must exist for each workspace member, so we can
store it directly and avoid going through the `.and_then()` when we need
to access it. This requires cloning the struct due to lack of
self-referential structs. An alternative would taking the `Project` from
`PyProjectToml` instead, but this could be confusing when passing the
`PyProjectToml` around.
2024-06-05 12:48:19 -04:00
Charlie Marsh ae9610104a
Avoid dropping `pip sync` requirements with markers (#4051)
## Summary

Thankfully this is pretty rare since `pip sync` is usually run on `pip
compile` output, and `pip compile` never outputs markers.

Closes https://github.com/astral-sh/uv/issues/4044
2024-06-05 16:05:46 +00:00
Zanie Blue b0d1fc85a9
Use standard path filters for `uv venv` tests (#4046)
Over in #4045 the existing filters were insufficient. We should use the
same strategy we use for our standard `TestContext` instead.
2024-06-05 10:25:10 -05:00
Zanie Blue f2dc08b242
Drop `git` dependency from `pip_entrypoints` test (#4049)
There's a release with the changes we want test coverage for now
2024-06-05 17:22:32 +02:00