Commit Graph

3835 Commits

Author SHA1 Message Date
Ben Beasley 80ab562f9f
In uv-globfilter, use the workspace fs-err in dev-dependencies (#9262)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

In uv-globfilter, use the workspace `fs-err` in `dev-dependencies`.

This fixes an unnecessary dev-dependency on `fs-err` 2.x even after the
workspace fs-err was updated to 3.x in
https://github.com/astral-sh/uv/pull/8625.

The `Cargo.lock` file still has `fs-err v2.11.0` after this PR, but it
is via `tracing-durations-export v0.3.0` rather than directly required
by any `uv` crate.

## Test Plan

<!-- How was it tested? -->

```
$ cd crates/uv-globfilter/
$ cargo test
```
2024-11-20 03:59:37 +00:00
Zanie Blue 7c4c15b879
Add trace logs to `is_extended_transient_error` (#9248)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-19 22:24:05 -05:00
Charlie Marsh f4799d2346
Add dedicated warning for empty stdin (#9256)
## Summary

I ran into this in practice:

![Screenshot 2024-11-19 at 4 20
37 PM](https://github.com/user-attachments/assets/d89aa9f1-828a-492e-af5a-3434e277580e)
2024-11-19 17:52:41 -05:00
Charlie Marsh 9fb7f81738
Catch retries with wrapped `reqwest` errors (#9253)
## Summary

It turns out that `WrappedReqwestError` skips the `reqwest::Error`
itself in order to hack the display. This PR adds it to the list of
variants we check when retrying transient errors.

Closes https://github.com/astral-sh/uv/issues/9246.


## Test Plan

Patched `reqwest` locally to return an error in `bytes()`. Verified that
it was _not_ caught prior to this PR, but was caught afterwards.
2024-11-19 17:08:48 -05:00
konsti 8e0389e2fd
Build backend: Build editable (#9230)
Support for editable installs. This is a simple PEP 660 implementation.
2024-11-19 21:52:11 +00:00
konsti 0913382aa5
Better WrappedReqwestError docs (#9251)
Document the hack with which we insert a hint message for offline cases.
2024-11-19 16:51:58 -05:00
Ed Morley 62e0ee7f67
Fix grammar in missing platform marker error message (#9240)
`s/a platform markers/a platform marker/`
2024-11-19 20:03:52 +00:00
Ahmed Ilyas 0454c3b3ec
Sort hashes in `uv export` output (#9237)
## Summary

Closes #9225 

## Test Plan

`cargo test`
2024-11-19 19:54:27 +00:00
Zanie Blue ca9aaf1c48
Reorganize the project concept documentation (#9121)
- Adds a collapsible section for the project concept
- Splits the project concept document into several child documents.
- Moves the workspace and dependencies documents to under the project
section
- Adds a mkdocs plugin for redirects, so links to the moved documents
still work

I attempted to make the minimum required changes to the contents of the
documents here. There is a lot of room for improvement on the content of
each new child document. For review purposes, I want to do that work
separately. I'd prefer if the review focused on this structure and idea
rather than the content of the files.

I expect to do this to other documentation pages that would otherwise be
very nested.

The project concept landing page and nav (collapsed by default) looks
like this now:

<img width="1507" alt="Screenshot 2024-11-14 at 11 28 45 AM"
src="https://github.com/user-attachments/assets/88288b09-8463-49d4-84ba-ee27144b62a5">
2024-11-19 13:52:12 -06:00
Charlie Marsh 45eeae61ff
Use existing index field in PEP 723 scripts (#9238)
## Summary

Purely internal, this setting already exists.
2024-11-19 14:45:12 -05:00
Hristo Filaretov f49230471c
Add manylinux target triples up to glibc 2.40 (#9234)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

PR #4965 added `*-manylinux_2_31` as a target triple, and issue #4966
described the need for a more general solution.

In lieu of a general solution, this PR adds further explicit manylinux
target triples for different glibc version up to the one used by the
latest Ubuntu release (glibc 2.40 used in Ubuntu 24.10).

## Test Plan

<!-- How was it tested? -->

Local, manual testing with a Python wheel targeting
`x86_64-manylinux_2_35`.
2024-11-19 14:37:43 -05:00
Charlie Marsh 56d3622087
Bump version to v0.5.3 (#9229) 2024-11-19 18:53:54 +00:00
renovate[bot] dae630ae50
Update Rust crate fs-err to v3 (#8625) 2024-11-19 15:09:00 +00:00
konsti 6d5c708082
Typo (#9222) 2024-11-19 12:01:17 +00:00
konsti 9460398371
Build backend: Support data files (#9197)
Allow including data files in wheels, configured through
`pyproject.toml`. This configuration is currently only read in the build
backend. We'd only start using it in the frontend when we're adding a
fast path.

Each data entry is a directory, whose contents are copied to the
matching directory in the wheel in
`<name>-<version>.data/(purelib|platlib|headers|scripts|data)`. Upon
installation, this data is moved to its target location, as defined by
<https://docs.python.org/3.12/library/sysconfig.html#installation-paths>:
- `data`: Installed over the virtualenv environment root. Warning: This
may override existing files!
- `scripts`: Installed to the directory for executables, `<venv>/bin` on
Unix or `<venv>\Scripts` on Windows. This directory is added to PATH
when the virtual environment is activated or when using `uv run`, so
this data type can be used to install additional binaries. Consider
using `project.scripts` instead for starting Python code.
- `headers`: Installed to the include directory, where compilers
building Python packages with this package as built requirement will
search for header files.
- `purelib` and `platlib`: Installed to the `site-packages` directory.
It is not recommended to uses these two options.

For simplicity, for now we're just defining a directory to be copied for
each data directory, while using the glob based include mechanism in the
background. We thereby introduce a third mechanism next to the main
includes and the PEP 639 mechanism, which is not what we should finalize
on.
2024-11-19 12:59:59 +01:00
Charlie Marsh 821f3de095
Automatically retry body errors when processing response (#9213)
## Summary

The reqwest middleware doesn't retry errors that occur "after" the
request completes -- but in some cases, these do include spurious errors
that we want to retry. See https://github.com/astral-sh/uv/issues/8144
for examples. This PR adds a second retry layer during the response
_handler_, which should help with some of the spurious failures we see
in the linked issue.

Closes https://github.com/astral-sh/uv/issues/8144.
2024-11-19 04:14:58 +00:00
Charlie Marsh 9106744503
Only install the specified project with `--frozen --package` in legacy non-`[project]` workspaces (#9215)
## Summary

We missed the case in which the user has a legacy non-`[project]` root
-- we were always installing all members.

Closes https://github.com/astral-sh/uv/issues/9214.
2024-11-19 03:18:00 +00:00
Zanie Blue b642dd9287
Update `uv tool install --force` to imply `--reinstall-package <name>` (#9074) 2024-11-18 19:41:50 -06:00
Charlie Marsh e4fc875afa
Allow conflicting extras in explicit index assignments (#9160)
## Summary

This PR enables something like the "final boss" of PyTorch setups --
explicit support for CPU vs. GPU-enabled variants via extras:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = []

[project.optional-dependencies]
cpu = [
    "torch==2.5.1+cpu",
]
gpu = [
    "torch==2.5.1",
]

[tool.uv.sources]
torch = [
    { index = "torch-cpu", extra = "cpu" },
    { index = "torch-gpu", extra = "gpu" },
]

[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true

[tool.uv]
conflicts = [
    [
        { extra = "cpu" },
        { extra = "gpu" },
    ],
]
```

It builds atop the conflicting extras work to allow sources to be marked
as specific to a dedicated extra being enabled or disabled.

As part of this work, sources now have an `extra` field. If a source has
an `extra`, it means that the source is only applied to the requirement
when defined within that optional group. For example, `{ index =
"torch-cpu", extra = "cpu" }` above only applies to
`"torch==2.5.1+cpu"`.

The `extra` field does _not_ mean that the source is "enabled" when the
extra is activated. For example, this wouldn't work:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.13.0"
dependencies = ["torch"]

[tool.uv.sources]
torch = [
    { index = "torch-cpu", extra = "cpu" },
    { index = "torch-gpu", extra = "gpu" },
]

[[tool.uv.index]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "torch-gpu"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
```

In this case, the sources would effectively be ignored. Extras are
really confusing... but I think this is correct? We don't want enabling
or disabling extras to affect resolution information that's _outside_ of
the relevant optional group.
2024-11-19 01:06:25 +00:00
Charlie Marsh d08bfee718
Remove separate test files in favor of same-file `mod tests` (#9199)
## Summary

These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
2024-11-18 20:11:46 +00:00
Charlie Marsh 747d69dc96
Respect `[[tool.uv.index]]` in PEP 723 scripts (#9208)
## Summary

There was a typo here, combined with a lack of test coverage.

Closes https://github.com/astral-sh/uv/issues/9201.
2024-11-18 13:19:31 -05:00
Ahmed Ilyas ad342009af
Better error message when `git` is not found (#9206)
## Summary

Closes https://github.com/astral-sh/uv/issues/9200


## Test Plan

Using the following Dockerfile:
```Dockerfile
FROM debian:latest

RUN apt-get update && apt-get install -y python3

WORKDIR /app
COPY target/debug/uv .
RUN chmod +x uv

RUN /app/uv venv && /app/uv pip install git@github.com:pallets/flask.git
```

```
❯ cargo build -q -p uv && docker build .
...
 => ERROR [6/6] RUN /app/uv venv && /app/uv pip install git@github.com:pallets/flask.git                    0.4s
------
 > [6/6] RUN /app/uv venv && /app/uv pip install git@github.com:pallets/flask.git:
0.275 Using CPython 3.11.2 interpreter at: /usr/bin/python3
0.275 Creating virtual environment at: .venv
0.318   × Failed to download and build `git @
0.318   │ file:///app/github.com:pallets/flask.git`
0.318   ├─▶ Git operation failed
0.318   ╰─▶ Git executable not found. Ensure that Git is installed and available.
------
Dockerfile:9
--------------------
   7 |     RUN chmod +x uv
   8 |
   9 | >>> RUN /app/uv venv && /app/uv pip install git@github.com:pallets/flask.git
  10 |
--------------------
ERROR: failed to solve: process "/bin/sh -c /
```
2024-11-18 12:41:22 -05:00
Charlie Marsh f1554c5ecd
Avoid multiple dependencies in `verify_hashes_mismatch` (#9207)
## Summary

Last time, I swear.
2024-11-18 17:24:55 +00:00
Charlie Marsh 0aaa6ba5be
Migrate to `zlib-rs` (#9184)
## Summary

I've tried this a few times; just curious if it passes tests.
2024-11-18 15:45:14 +00:00
Charlie Marsh 46151c13cd
Fix flaky failure in `verify_hashes_mismatch` (#9198)
## Summary

I moved this to a separate test. The packages may or may not be
downloaded already, since the previous command fails -- it just depends
on timing.
2024-11-18 14:26:53 +00:00
konsti a07d5a1d18
Update pubgrub to `Ranges::from_iter` (#9145)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-11-18 12:28:17 +00:00
Shantanu 587c9b02a6
Fix typo in environment variable name (#9186)
https://github.com/astral-sh/uv/pull/9170/files#r1845768798
2024-11-18 02:54:26 +00:00
Charlie Marsh 8d258655e2
Sort distributions when computing hash (#9185)
## Summary

The distributions used to be stored in a `BTreeMap`, keyed by name.
They're now stored in a graph... so iteration isn't guaranteed to
produce a deterministic hash!

This fixes a "flaky" test, though it's actually a real bug. The test was
right!

Closes #9137.
2024-11-17 21:43:03 -05:00
Charlie Marsh a72e2f9195 Avoid flaky preparedness error in verify_hashes_mismatch 2024-11-17 21:39:40 -05:00
Shantanu 71d9c45393
Turn `--verify-hashes` on by default (#9170)
Fixes #9164

Using clap's `default_value_t` makes the `flag` function unhappy, so
just set the default when we unwrap. Tested with no flags,
`--verify-hashes`, `--no-verify-hashes` and setting in uv.toml

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 01:57:54 +00:00
Charlie Marsh 5ba186628b
Avoid showing disjoint marker error with `true` (#9169)
## Summary

Ran into this in practice, and it's awkward:

![Screenshot 2024-11-16 at 12 19
25 PM](https://github.com/user-attachments/assets/1c5d296e-b6ea-4e28-83fc-0ae096dfe2b8)
2024-11-17 20:50:44 -05:00
renovate[bot] 352e3341de
Update Rust crate flate2 to v1.0.35 (#9176) 2024-11-17 20:48:21 -05:00
Charlie Marsh 12266f8f81
Add a dedicated error for `include = "dev"` with `tool.uv.dev-dependencies` (#9173)
## Summary

This isn't really spec-compliant, so we already don't allow it -- this
just adds a better error message and an explicit test for it.
2024-11-17 13:22:04 -05:00
Charlie Marsh fb3f365d10
Support overrides and constraints in PEP 723 scripts (#9162)
## Summary

Closes https://github.com/astral-sh/uv/issues/9141.
2024-11-15 23:03:39 -05:00
Charlie Marsh e21227504b
Include cyan for prefix on versions (#9155)
## Summary

The `v` in `v1.0.0` wasn't cyan, but the version was. This wasn't
obvious from the snapshots, which ignore color.
2024-11-15 16:11:19 -05:00
Charlie Marsh 5bff2ba243
Include extras and dependency groups in derivation chains (#9113)
## Summary

Displays extras and dependency groups when explaining inclusion.
2024-11-15 15:37:18 -05:00
Charlie Marsh 8dd095cab8
Include version constraints in derivation chains (#9112)
## Summary

Derivation chains can now include the versions at which a package was
requested.
2024-11-15 15:06:24 -05:00
renovate[bot] 312ae12650
Update Rust crate thiserror to v2 (#9006) 2024-11-15 09:54:16 -06:00
konsti 21d570fac9
Build backend: Include readme and license files (#9149)
When building source distributions, we need to include the readme, so it
can become part the METADATA body when building the wheel. We also need
to support the license files from PEP 639. When building the source
distribution, we copy those file relative to their origin, when building
the wheel, we copy them to `.dist-info/licenses`.

The test for idempotence in wheel building is merged into the file
listing test, which also covers that source tree -> source dist -> wheel
is equivalent to source tree -> wheel, though we do need to check for
file inclusion stronger here.

Best reviewed commit-by-commit
2024-11-15 14:41:39 +00:00
konsti 997ff9d57a
Allow apostrophe in venv name (#8984)
Escape an apostrophe in the venv path name.

Fixes #8947
2024-11-15 10:52:10 +01:00
konsti 0abb2a4595
Refactor shell quoting (#9055)
Move the shlex-like quoting utils in the uv-shell crate, so we only
write `r#"'"'"'"#` once.

Split out from #8984
2024-11-15 09:06:54 +00:00
Micha Reiser 7b4197bc0e
Detect nested workspace inside the current workspace and members with identical names (#9094)
## Summary

Align uv's workspace discovery with red knots (see
https://github.com/astral-sh/ruff/pull/14308#issuecomment-2474296308)

* Detect nested workspace inside the current workspace rather than
testing if the current workspace is a member of any outer workspace.
* Detect packages with identical names.

## Test Plan

I added two integration tests. I also back ported the tests to main to
verify that both these invalid workspaces aren't catched by uv today.
That makes this, technically, a breaking change but I would consider the
change a bug fix.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-15 04:03:00 +00:00
Charlie Marsh 2b08d767cd
Show derivation markers for resolutions with project name (#9136)
## Summary

I was wrongly using `.name()` to detect if a package was "not root", but
in `pip compile`, the root can have a name -- so we were failing to find
the derivation chain.
2024-11-14 19:25:25 -05:00
Zanie Blue 195f4b634f
Bump version to 0.5.2 (#9133) 2024-11-14 15:59:50 -06:00
Charlie Marsh 92735ced9a
Misc. touch-ups to target-to-resolution routine (#9131) 2024-11-14 16:24:03 -05:00
Charlie Marsh fe477c3417
Show full derivation chain when encountering build failures (#9108)
## Summary

This PR adds context to our error messages to explain _why_ a given
package was included, if we fail to download or build it.

It's quite a large change, but it motivated some good refactors and
improvements along the way.

Closes https://github.com/astral-sh/uv/issues/8962.
2024-11-14 15:48:26 -05:00
Charlie Marsh a552f74308
Refactor `Resolution` type to retain dependency graph (#9106)
## Summary

This PR should not contain any user-visible changes, but the goal is to
refactor the `Resolution` type to retain a dependency graph. We want to
be able to explain _why_ a given package was excluded on error (see:
https://github.com/astral-sh/uv/issues/8962), which in turn requires
that at install time, we can go back and figure out the dependency
chain. At present, `Resolution` is just a map from package name to
distribution; this PR remodels it as a graph in which each node is a
package, and the edges contain markers plus extras or dependency groups.
2024-11-14 15:25:34 -05:00
konsti 3a7db17147
Build backend: Add source tree -> source dist -> wheel tests (#9091)
A first milestone: source tree -> source dist -> wheel -> install works.
This PR adds a test for this.

There's obviously a lot still missing, including basics such as the
Readme inclusion.
2024-11-14 19:15:32 +00:00
konsti 9a20f8c7b7
Remove end-of-line whitespace in snapshots (#9127)
I've configured my IDE to remove trailing end-of-line whitespace, and
these snapshots were causing trouble.
2024-11-14 18:16:20 +00:00
konsti c2ccc2c5dc
Support `copy_dir_all` in unit tests too (#9124)
Refactoring for https://github.com/astral-sh/uv/pull/9091
2024-11-14 18:05:15 +00:00