Commit Graph

83 Commits

Author SHA1 Message Date
Charlie Marsh c5032aee80
Bump MSRV to 1.85 and Edition 2024 (#13516)
## Summary

Builds on https://github.com/astral-sh/uv/pull/11724.

Closes https://github.com/astral-sh/uv/issues/13476.
2025-05-18 19:38:43 -04:00
konsti 5d37c7ecc5
Apply first set of Rustfmt edition 2024 changes (#13478)
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.

Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
2025-05-16 20:19:02 -04:00
Charlie Marsh 42dcea0ee2
Bump MSRV to 1.84 (#12670)
## Summary

Closes https://github.com/astral-sh/uv/issues/12649.
2025-04-04 11:49:26 -04:00
Charlie Marsh e4c98e976f
Use `Box<Path>` in lieu of `PathBuf` for immutable structs (#12346)
## Summary

I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
2025-03-25 21:56:06 +00:00
Charlie Marsh 46967723bb
Move lowered requirement source type out of `uv-pypi-types` (#12356)
## Summary

This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
2025-03-20 21:16:12 -04:00
Charlie Marsh e40c551b80
Use resolver-returned wheel over alternate cached wheel (#12301)
## Summary

I think this is reasonable to change. Right now, if you're on Python
3.11, the resolver returns `multiprocess-0.70.17-py311-none-any.whl`,
but `multiprocess-0.70.17-py310-none-any.whl` is in the cache, we'll
reuse `multiprocess-0.70.17-py310-none-any.whl` (since it _is_
compatible with Python 3.11).

Instead, we now _require_ the cached wheel to match the wheel returned
by the resolver.

Closes https://github.com/astral-sh/uv/issues/12273.
2025-03-18 21:41:04 -04:00
Aria Desires ada1acb32f
Make cache errors non-fatal in Planner::build (#12281)
Same basic approach as #11105, including a cache version bump.

Fixes #12274
2025-03-18 11:27:21 -04:00
Charlie Marsh 72be5ffb25
Always reinstall local source trees passed to `uv pip install` (#12176)
## Summary

This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)

Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.

Closes #12038.
2025-03-17 21:12:21 +00:00
Ankit Saini fb35875f24
Use hash instead of full wheel name in wheels bucket (#11738)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->


## Summary
Closes #2410 
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the name of files in `wheels` bucket to use a hash instead
of the wheel name as to not exceed maximum file length limit on various
systems.

This only addresses the primary concern of #2410. It still does _not_
address:
- Path limit of 260 on windows:
https://github.com/astral-sh/uv/issues/2410#issuecomment-2062020882
To solve this we need to opt-in to longer path limits on windows
([ref](https://github.com/astral-sh/uv/issues/2410#issuecomment-2150532658)),
but I think that is a separate issue and should be a separate MR.
- Exceeding filename limit while building a wheel from source
distribution
As per my understanding, this is out of uv's control. Name of the output
wheel will be decided by build-backend used by the project. For wheels
built from source distribution, pip also uses the wheel names in cache.
So I have not touched `sdists` cache.


I have added a `filename: WheelFileName` field in `Archive`, so we can
use it while indexing instead of relying on the filename on disk.
Another way to do this was to read `.dist-info/WHEEL` and
`.dist-info/METADATA` and build `WheelFileName` but that seems less
robust and will be slower.
## Test Plan

<!-- How was it tested? -->
Tested by installing `yt-dlp`, `httpie` and `sqlalchemy` and verifying
that cache files in `wheels` bucket use hash.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-02-26 22:41:57 +00:00
konsti ae916cff5a
Reduce overhead in converting resolutions (#11660)
Solving spent a chunk of its time just converting resolutions, the left
two blocks:


![image](https://github.com/user-attachments/assets/6f266440-c6e2-447c-ad7f-f92244f9d09b)

These blocks are `ResolverOutput::from_state` with 1.3% and
`ForkState::into_resolution` with 4.1% of resolver thread runtime for
apache airflow universal.

We reduce the overhead spent in those functions, to now 1.1% and 2.1% of
resolver time spend in those functions by:

Commit 1: Replace the hash set for the edges with a vec in
`ForkState::into_resolution`. We deduplicate edges anyway when
collecting them, and the hash-and-insert was slow.

Commit 2: Reduce the distribution clonign in
`ResolverOutput::from_state` by using an `Arc`.

The same profile excerpt for the resolver with the branch (note that
there is now an unrelated block between the two we optimized):


![image](https://github.com/user-attachments/assets/e36c205d-2cf8-4fe6-a2dd-3020c0515922)

Wall times are noisy, but the profiles show those changes as
improvements.

```
$ hyperfine --warmup 2 "./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal" "./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal"
Benchmark 1: ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
  Time (mean ± σ):      99.1 ms ±   3.8 ms    [User: 111.8 ms, System: 115.5 ms]
  Range (min … max):    93.6 ms … 110.4 ms    29 runs
 
Benchmark 2: ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal
  Time (mean ± σ):      97.1 ms ±   4.3 ms    [User: 114.8 ms, System: 112.0 ms]
  Range (min … max):    90.9 ms … 112.4 ms    29 runs
 
Summary
  ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal ran
    1.02 ± 0.06 times faster than ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
```
2025-02-20 12:13:01 -08:00
Charlie Marsh 306fcfe718
Use consistent 'Registry' prefix for wheel and source distribution logs (#11270)
## Summary

We say "Registry requirement already cached" for source distributions,
but for wheels, it's just "Requirement already cached".
2025-02-06 01:35:11 +00:00
Charlie Marsh 1372c4e6de
Include `commit_id` and `requested_revision` in `direct_url.json` (#10862)
## Summary

Closes #3014.

Closes #10846.
2025-01-22 12:16:49 -05:00
Charlie Marsh b6aa40b29d
Read cached registry distributions when `--config-settings` are present (#10578)
## Summary

Closes https://github.com/astral-sh/uv/issues/10577.
2025-01-13 21:25:12 +00:00
Charlie Marsh c946e472fb
Don't treat `setuptools` and `wheel` as seed packages in uv sync on Python 3.12 (#10572)
## Summary

Closes https://github.com/astral-sh/uv/issues/10566.
2025-01-13 12:43:11 -05:00
Charlie Marsh 0d57d298e6
Avoid downgrading packages when `--upgrade` is provided (#10097)
## Summary

When `--upgrade` is provided, we should retain already-installed
packages _if_ they're newer than whatever is available from the
registry.

Closes https://github.com/astral-sh/uv/issues/10089.
2025-01-06 17:41:43 -05:00
Charlie Marsh 0e836bcbbc
Include both installed and request in satisfied logging (#10100) 2024-12-22 20:41:28 +00:00
Charlie Marsh f1ccbcb065
Consistently enforce requested-vs.-built metadata when retrieving wheels (#9484)
## Summary

We were missing a bunch of edge cases, e.g., the wheel exists in the
cache already.

Closes https://github.com/astral-sh/uv/issues/9480.
2024-11-27 16:51:01 -05:00
Charlie Marsh 8c663d2a3f
Convert to `RequirementSource` rather than `Requirement` (#9107)
## Summary

This seems like a vestige left over from a refactor.
2024-11-13 20:00:13 -05:00
Charlie Marsh 9339e55a11
Add `version` to `ResolvedDist` (#9102)
## Summary

I need this for the derivation chain work
(https://github.com/astral-sh/uv/issues/8962), but it just seems
generally useful. You can't always get a version from a `Dist` (it could
be URL-based!), but when we create a `ResolvedDist`, we _do_ know the
version (and not just the URL). This PR preserves it.
2024-11-13 19:06:16 -05:00
Charlie Marsh 5b391770df
Add support for named and explicit indexes (#7481)
## Summary

This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.

Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```

You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.

### Index priority

Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.

Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.

You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.

### Index pinning

If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"

[tool.uv.sources]
torch = { index = "pytorch" }
```

If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true

[tool.uv.sources]
torch = { index = "pytorch" }
```

Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.

(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)

### Default index

By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```

### Name reuse

If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.

For example, given:

```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"

[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```

The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.

Closes #171.

## Future work

- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
  { index = "cpu", markers = "sys_platform == 'darwin'" },
  { index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
2024-10-15 18:24:23 -04:00
Charlie Marsh 14507a1793
Add `uv-` prefix to all internal crates (#7853)
## Summary

Brings more consistency to the repo and ensures that all crates
automatically show up in `--verbose` logging.
2024-10-01 20:15:32 -04:00
Charlie Marsh b6ce39f45e
Remove lossy resolution-to-requirements conversion in install plan (#7798)
## Summary

This is a longstanding piece of technical debt. After we resolve, we
have a bunch of `ResolvedDist` entries. We then convert those to
`Requirement` (which is lossy -- we lose information like "the index
that the package was resolved to"), and then back to `Dist`.
2024-09-30 10:13:09 -04:00
Charlie Marsh 66d7ec541a
Avoid reusing cached downloaded binaries with `--no-binary` (#7772)
## Summary

Historically, we've allowed the use of wheels that were downloaded from
PyPI even when the user passes `--no-binary`, if the wheel exists in the
cache. This PR modifies the cache lookup code such that we respect
`--no-build` and `--no-binary` in those paths.

Closes https://github.com/astral-sh/uv/issues/2154.
2024-09-29 17:34:52 +00:00
Charlie Marsh f895c40a4e
Avoid removing seed packages for `uv venv --seed` environments (#7410)
## Summary

Closes https://github.com/astral-sh/uv/issues/7121.
2024-09-15 22:27:52 +00:00
Charlie Marsh a8bd0211e0
Invalidate cache when `--config-settings` change (#7139)
## Summary

If `--config-settings` are provided, we cache the built wheels under one
more subdirectory.

We _don't_ invalidate the actual source (i.e., trigger a re-download) or
metadata, though -- those can be reused even when `--config-settings`
change.

Closes https://github.com/astral-sh/uv/issues/7028.
2024-09-10 01:49:16 +00:00
Charlie Marsh 4f2349119c
Add support for dynamic cache keys (#7136)
## Summary

This PR adds a more flexible cache invalidation abstraction for uv, and
uses that new abstraction to improve support for dynamic metadata.

Specifically, instead of relying solely on a timestamp, we now pass
around a `CacheInfo` struct which (as of now) contains
`Option<Timestamp>` and `Option<Commit>`. The `CacheInfo` is saved in
`dist-info` as `uv_cache.json`, so we can test already-installed
distributions for cache validity (along with testing _cached_
distributions for cache validity).

Beyond the defaults (`pyproject.toml`, `setup.py`, and `setup.cfg`
changes), users can also specify additional cache keys, and it's easy
for us to extend support in the future. Right now, cache keys can either
be instructions to include the current commit (for `setuptools_scm` and
similar) or file paths (for `hatch-requirements-txt` and similar):

```toml
[tool.uv]
cache-keys = [{ file = "requirements.txt" }, { git = true }]
```

This change should be fully backwards compatible.

Closes https://github.com/astral-sh/uv/issues/6964.

Closes https://github.com/astral-sh/uv/issues/6255.

Closes https://github.com/astral-sh/uv/issues/6860.
2024-09-09 20:19:15 +00:00
Charlie Marsh 56cc0c9b3c
Avoid using editable tag in lockfile for non-package dependencies (#6728)
## Summary

Use a dedicated source type for non-package requirements. Also enables
us to support non-package `path` dependencies _and_ removes the need to
have the member `pyproject.toml` files available when we sync _and_
makes it explicit which dependencies are virtual vs. not (as evidenced
by the snapshot changes). All good things!
2024-08-28 01:19:05 +00:00
Charlie Marsh a7850d2a1c
Use separate types to represent raw vs. resolver markers (#6646)
## Summary

This is similar to https://github.com/astral-sh/uv/pull/6171 but more
expansive... _Anywhere_ that we test requirements for platform
compatibility, we _need_ to respect the resolver-friendly markers. In
fixing the motivating issue (#6621), I also realized that we had a bunch
of bugs here around `pip install` with `--python-platform` and
`--python-version`, because we always performed our `satisfy` and `Plan`
operations on the interpreter's markers, not the adjusted markers!

Closes https://github.com/astral-sh/uv/issues/6621.
2024-08-26 18:00:21 +00:00
Charlie Marsh f7835243c5
Only use relative paths in lockfile (#6490)
For users who were using absolute paths in the `pyproject.toml`
previously, this is a behavior change: We now convert all absolute paths
in `path` entries to relative paths. Since i assume that no-one relies
on absolute path in their lockfiles - they are intended to be portable -
I'm tagging this as a bugfix.

Closes https://github.com/astral-sh/uv/pull/6438
Fixes https://github.com/astral-sh/uv/issues/6371
2024-08-23 22:19:10 -04:00
Charlie Marsh 611a9003c9
Don't canonicalize paths to user requirements (#6560) 2024-08-24 02:02:14 +00:00
Charlie Marsh 19ac9af167
Use consistent canonicalization for URLs (#5980)
Right now, the URL gets out-of-sync with the install path, since the
install path is canonicalized. This leads to a subtle error on Windows
(in CI) in which we don't preserve caching across resolution and
installation.
2024-08-09 21:43:36 -04:00
konsti fcbee9ce25
Support relative path wheels (#5969)
Surprisingly, this is a lockfile schema change: We can't store relative
paths in urls, so we have to store a `filename` entry instead of the
whole url.

Fixes #4355
2024-08-09 21:57:16 +00:00
Charlie Marsh 21408c1f35
Enforce extension validity at parse time (#5888)
## Summary

This PR adds a `DistExtension` field to some of our distribution types,
which requires that we validate that the file type is known and
supported when parsing (rather than when attempting to unzip). It
removes a bunch of extension parsing from the code too, in favor of
doing it once upfront.

Closes https://github.com/astral-sh/uv/issues/5858.
2024-08-08 21:39:47 -04:00
Charlie Marsh e4ec6e4025
Avoid panic when re-locking with precise commit (#5863)
## Summary

Very subtle bug. The scenario is as follows:

- We resolve: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git" }`

- The user then changes the request to: `elmer-circuitbuilder = { git =
"https://github.com/ElmerCSC/elmer_circuitbuilder.git", rev =
"44d2f4b19d6837ea990c16f494bdf7543d57483d" }`

- When we go to re-lock, we note two facts:

1. The "default branch" resolves to
`44d2f4b19d6837ea990c16f494bdf7543d57483d`.
2. The metadata for `44d2f4b19d6837ea990c16f494bdf7543d57483d` is
(whatever we grab from the lockfile).

- In the resolver, we then ask for the metadata for
`44d2f4b19d6837ea990c16f494bdf7543d57483d`. It's already in the cache,
so we return it; thus, we never add the
`44d2f4b19d6837ea990c16f494bdf7543d57483d` ->
`44d2f4b19d6837ea990c16f494bdf7543d57483d` mapping to the Git resolver,
because we never have to resolve it.

This would apply for any case in which a requested tag or branch was
replaced by its precise SHA. Replacing with a different commit is fine.

It only applied to `tool.uv.sources`, and not PEP 508 URLs, because the
underlying issue is that we aren't consistent about "automatically"
extracting the precise commit from a Git reference.

Closes https://github.com/astral-sh/uv/issues/5860.
2024-08-07 10:56:15 -04:00
Charlie Marsh 51b7e9bff1
Retain editable designation for cached wheel installs (#5545)
## Summary

The package was being installed as editable, but it wasn't marked as
such in `uv pip list`, as the `direct-url.json` was wrong.

Closes https://github.com/astral-sh/uv/issues/5543.
2024-07-28 22:39:48 -04:00
Charlie Marsh d798bb3973
Avoid always rebuilding dynamic metadata (#5206)
## Summary

I don't think that "always reinstall" is tenable for `uv run`. My
perspective on this is that if you want "always reinstall", you can now
set it persistently in your `pyproject.toml` or `uv.toml`.

As a smaller change, we could instead disable this _only_ for the
Project API.

Closes https://github.com/astral-sh/uv/issues/4946.
2024-07-22 00:04:03 +00:00
Zanie Blue dd7da6af5f
Change "toolchain" to "python" (#4735)
Whew this is a lot.

The user-facing changes are:

- `uv toolchain` to `uv python` e.g. `uv python find`, `uv python
install`, ...
- `UV_TOOLCHAIN_DIR` to` UV_PYTHON_INSTALL_DIR`
- `<UV_STATE_DIR>/toolchains` to `<UV_STATE_DIR>/python` (with
[automatic
migration](https://github.com/astral-sh/uv/pull/4735/files#r1663029330))
- User-facing messages no longer refer to toolchains, instead using
"Python", "Python versions" or "Python installations"

The internal changes are:

- `uv-toolchain` crate to `uv-python`
- `Toolchain` no longer referenced in type names
- Dropped unused `SystemPython` type (previously replaced)
- Clarified the type names for "managed Python installations"
- (more little things)
2024-07-03 07:44:29 -05:00
Charlie Marsh ac87fd4006
Disable Clippy's `too-many-arguments` rule (#4663)
## Summary

We allow this constantly, I think it's just too pedantic for us.
2024-06-30 19:30:38 +00:00
renovate[bot] 3251690327
Update Rust crate rustc-hash to v2 (#4461) 2024-06-24 01:13:05 +00:00
Zanie Blue 5a007b6b9f
Add `BuildOptions` for centralized combination of `NoBuild` and `NoBinary` (#4284)
As requested in review of https://github.com/astral-sh/uv/pull/4067
2024-06-12 21:33:33 +00:00
Zanie Blue 1ab4041baa
Allow specific `--only-binary` and `--no-binary` packages to override `:all:` (#4067)
Updates `--no-binary <package>` to take precedence over `--only-binary
:all:` and `--only-binary <package>` to take precedence over
`--no-binary :all:`.

I'm not entirely sure about this behavior, e.g. maybe I provided
`--only-binary :all:` later on the command line and really want it to
override those earlier arguments of `--no-binary <package>` for safety.
Right now we just fail to solve though since we can't satisfy the
overlapping requests.

Closes https://github.com/astral-sh/uv/issues/4063
2024-06-12 15:47:45 -05:00
Charlie Marsh d8f1de6134
Use separate path types for directories and files (#4285)
## Summary

This is what I consider to be the "real" fix for #8072. We now treat
directory and path URLs as separate `ParsedUrl` types and
`RequirementSource` types. This removes a lot of `.is_dir()` forking
within the `ParsedUrl::Path` arms and makes some states impossible
(e.g., you can't have a `.whl` path that is editable). It _also_ fixes
the `direct_url.json` for direct URLs that refer to files. Previously,
we wrote out to these as if they were installed as directories, which is
just wrong.
2024-06-12 15:59:21 -04:00
konsti 44833801b3
Support locking relative paths (#4205)
By splitting `path` into a lockable, relative (or absolute) and an
absolute installable path and by splitting between urls and paths by
dist type, we can store relative paths in the lockfile.
2024-06-11 11:58:03 +00:00
Zanie Blue 325982c418
Rename `uv-interpreter` crate to `uv-toolchain` (#4120)
In preparation for managed toolchains #2607, just renames the crate to
something broader.

See #4121 and https://github.com/astral-sh/uv/pull/4138 to see the final
intent.
2024-06-07 13:59:14 -05:00
konsti 01d1a39c21
Add `uv run --package` (#3864)
Add a `--package` option that allows switching the current project in
the workspace. Wherever you are in a workspace, you should be able to
run with any other project as root. This is the uv equivalent of `cargo
run -p`.

I don't love the `--package` name, esp. since `-p` is already taken and
in general to many things start with p already.

Part of this change is moving the workspace discovery of
`ProjectWorkspace` to `Workspace` itself.

## Usage

In albatross-virtual-workspace:

```console
$ uv venv
$ uv run --preview --package bird-feeder python -c "import albatross"
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds
Built 2 editables in 167ms
Resolved 5 packages in 4ms
Installed 5 packages in 1ms
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds)
 + sniffio==1.3.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'albatross'
$ uv venv
$ uv run --preview --package albatross python -c "import albatross"
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/albatross
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds
Built 3 editables in 173ms
Resolved 7 packages in 6ms
Installed 7 packages in 1ms
 + albatross==0.1.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/albatross)
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-virtual-workspace/packages/seeds)
 + sniffio==1.3.1
 + tqdm==4.66.4
```

In albatross-root-workspace:

```console
$ uv venv
$ uv run --preview --package bird-feeder python -c "import albatross"
  Using Python 3.12.3 interpreter at: /home/konsti/.local/bin/python3
  Creating virtualenv at: .venv
  Activate with: source .venv/bin/activate
      Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
       Running `/home/konsti/projects/uv/target/debug/uv run --preview --package bird-feeder python -c 'import albatross'`
     Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder
     Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds                                              Built 2 editables in 161ms
  Resolved 5 packages in 4ms
  Installed 5 packages in 1ms
   + anyio==4.4.0
   + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder)
   + idna==3.6
   + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds)
   + sniffio==1.3.1
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named 'albatross'
$ uv venv
$ cargo run run --preview --package albatross python -c "import albatross"
Using Python 3.12.3 interpreter at: /home/konsti/.local/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `/home/konsti/projects/uv/target/debug/uv run --preview --package albatross python -c 'import albatross'`
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder
   Built file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds
Built 3 editables in 168ms
Resolved 7 packages in 5ms
Installed 7 packages in 1ms
 + albatross==0.1.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace)
 + anyio==4.4.0
 + bird-feeder==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/bird-feeder)
 + idna==3.6
 + seeds==1.0.0 (from file:///home/konsti/projects/uv/scripts/workspaces/albatross-root-workspace/packages/seeds)
 + sniffio==1.3.1
 + tqdm==4.66.4
```
2024-06-02 21:42:14 +00:00
konsti 081f20c53e
Add support for `tool.uv` into distribution building (#3904)
With the change, we remove the special casing of workspace dependencies
and resolve `tool.uv` for all git and directory distributions. This
gives us support for non-editable workspace dependencies and path
dependencies in other workspaces. It removes a lot of special casing
around workspaces. These changes are the groundwork for supporting
`tool.uv` with dynamic metadata.

The basis for this change is moving `Requirement` from
`distribution-types` to `pypi-types` and the lowering logic from
`uv-requirements` to `uv-distribution`. This changes should be split out
in separate PRs.

I've included an example workspace `albatross-root-workspace2` where
`bird-feeder` depends on `a` from another workspace `ab`. There's a
bunch of failing tests and regressed error messages that still need
fixing. It does fix the audited package count for the workspace tests.
2024-05-31 02:42:03 +00:00
Charlie Marsh 1fc6a59707
Remove special-casing for editable requirements (#3869)
## Summary

There are a few behavior changes in here:

- We now enforce `--require-hashes` for editables, like pip. So if you
use `--require-hashes` with an editable requirement, we'll reject it. I
could change this if it seems off.
- We now treat source tree requirements, editable or not (e.g., both `-e
./black` and `./black`) as if `--refresh` is always enabled. This
doesn't mean that we _always_ rebuild them; but if you pass
`--reinstall`, then yes, we always rebuild them. I think this is an
improvement and is close to how editables work today.

Closes #3844.

Closes #2695.
2024-05-28 15:49:34 +00:00
Charlie Marsh f3965fef5e
Use filename trait for `WheelWire` conversion (#3651)
## Summary

The main motivation here is that the `.filename()` method that we
implement on `Url` will do URL decoding for the last segment, which we
were missing here.

The errors are a bit awkward, because in
`crates/uv-resolver/src/lock.rs`, we wrap in `failed to extract filename
from URL: {url}`, so in theory we want the underlying errors to _omit_
the URL? But sometimes they use `#[error(transparent)]`?
2024-05-20 09:25:31 -04:00
Charlie Marsh 1124df9bc5
Remove subdirectory from direct wheel URL type (#3667)
## Summary

Closes #3665.
2024-05-20 02:01:57 +00:00
Ibraheem Ahmed 39af09f09b
Parallelize resolver (#3627)
## Summary

This PR introduces parallelism to the resolver. Specifically, we can
perform PubGrub resolution on a separate thread, while keeping all I/O
on the tokio thread. We already have the infrastructure set up for this
with the channel and `OnceMap`, which makes this change relatively
simple. The big change needed to make this possible is removing the
lifetimes on some of the types that need to be shared between the
resolver and pubgrub thread.

A related PR, https://github.com/astral-sh/uv/pull/1163, found that
adding `yield_now` calls improved throughput. With optimal scheduling we
might be able to get away with everything on the same thread here.
However, in the ideal pipeline with perfect prefetching, the resolution
and prefetching can run completely in parallel without depending on one
another. While this would be very difficult to achieve, even with our
current prefetching pattern we see a consistent performance improvement
from parallelism.

This does also require reverting a few of the changes from
https://github.com/astral-sh/uv/pull/3413, but not all of them. The
sharing is isolated to the resolver task.

## Test Plan

On smaller tasks performance is mixed with ~2% improvements/regressions
on both sides. However, on medium-large resolution tasks we see the
benefits of parallelism, with improvements anywhere from 10-50%.

```
./scripts/requirements/jupyter.in
Benchmark 1: ./target/profiling/baseline (resolve-warm)
  Time (mean ± σ):      29.2 ms ±   1.8 ms    [User: 20.3 ms, System: 29.8 ms]
  Range (min … max):    26.4 ms …  36.0 ms    91 runs
 
Benchmark 2: ./target/profiling/parallel (resolve-warm)
  Time (mean ± σ):      25.5 ms ±   1.0 ms    [User: 19.5 ms, System: 25.5 ms]
  Range (min … max):    23.6 ms …  27.8 ms    99 runs
 
Summary
  ./target/profiling/parallel (resolve-warm) ran
    1.15 ± 0.08 times faster than ./target/profiling/baseline (resolve-warm)
```
```
./scripts/requirements/boto3.in   
Benchmark 1: ./target/profiling/baseline (resolve-warm)
  Time (mean ± σ):     487.1 ms ±   6.2 ms    [User: 464.6 ms, System: 61.6 ms]
  Range (min … max):   480.0 ms … 497.3 ms    10 runs
 
Benchmark 2: ./target/profiling/parallel (resolve-warm)
  Time (mean ± σ):     430.8 ms ±   9.3 ms    [User: 529.0 ms, System: 77.2 ms]
  Range (min … max):   417.1 ms … 442.5 ms    10 runs
 
Summary
  ./target/profiling/parallel (resolve-warm) ran
    1.13 ± 0.03 times faster than ./target/profiling/baseline (resolve-warm)
```
```
./scripts/requirements/airflow.in 
Benchmark 1: ./target/profiling/baseline (resolve-warm)
  Time (mean ± σ):     478.1 ms ±  18.8 ms    [User: 482.6 ms, System: 205.0 ms]
  Range (min … max):   454.7 ms … 508.9 ms    10 runs
 
Benchmark 2: ./target/profiling/parallel (resolve-warm)
  Time (mean ± σ):     308.7 ms ±  11.7 ms    [User: 428.5 ms, System: 209.5 ms]
  Range (min … max):   287.8 ms … 323.1 ms    10 runs
 
Summary
  ./target/profiling/parallel (resolve-warm) ran
    1.55 ± 0.08 times faster than ./target/profiling/baseline (resolve-warm)
```
2024-05-17 11:47:30 -04:00