Commit Graph

180 Commits

Author SHA1 Message Date
konsti 4d34b28657
Set 4MB stack size for all threads, introduce `UV_STACK_SIZE` (#12839)
See #12769 for the motivation. We set the 4MB not only for the main
thread, but also for all tokio and rayon threads to fix a stack overflow
while unpacking wheels in production on Windows.

There are two variables for setting the stack size: A new
`UV_STACK_SIZE` that takes precedent, and the existing `RUST_MIN_STACK`.
When setting the stack size, `UV_STACK_SIZE` should be preferred, since
`RUST_MIN_STACK` affects all Rust applications, including build backends
we call (e.g., maturin). The minimum stack size is set to 1MB, the
lowest stack size we observed on a platform (Windows main thread).

Fixes #12769

## Test Plan

Tested manually with the example from #12769
2025-04-16 09:27:46 +02:00
konsti 88cd7d619f
Cache `which git` in `uv init` (#12893)
Avoid running `which` multiple times, to be more coherent with the other
git code. In preparation of improving the `uv init` git handling.
2025-04-15 09:40:08 +00:00
konsti 173d6f73d3
Add `uv-build` and `uv_build` aliases to `uv init --build-backend` (#12776)
This matches similar aliases for pdm/pdm-backend and hatch/hatchling.
2025-04-09 12:33:38 +00:00
Aria Desires c6e6478f66
fix parsing of `python-platform` in settings tomls (#12592)
serde needs to be told where to put underscores. someone clearly noticed
this when adding attributes for schemars, but they need to be present
for serde too and then schemars gets them for free.

Strictly speaking this would be a breaking change for anyone who noticed
the parsing was messed up and worked around it. So we add aliases for
backcompat, at least for a few releases.

Fixes #12590
2025-04-01 09:08:35 -04:00
Charlie Marsh 175017bf51
Bump least-recent non-EOL macOS version to 13.0 (#12518)
## Summary

We use the least-recent non-EOL macOS version by default, and this has
since changed (i.e., macOS 12 went EOL).

Closes https://github.com/astral-sh/uv/issues/12487.
2025-03-28 07:49:45 -04:00
Charlie Marsh 6e03adc956
Fix schemars macro usage in build backend enum (#12503)
## Test Plan

```
cargo test --profile fast-build -p uv-scripts
```
2025-03-27 02:23:21 +00:00
Charlie Marsh e4c98e976f
Use `Box<Path>` in lieu of `PathBuf` for immutable structs (#12346)
## Summary

I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
2025-03-25 21:56:06 +00:00
Charlie Marsh 149102a4e7
Use a boxed slice for extras and groups (#12391)
## Summary

A very common struct, and these are immutable. Easy to optimize.
2025-03-22 11:53:36 -04:00
Ahmed Ilyas a80353de2b
Add `uv sync --check` flag (#12342)
## Summary

Closes #12338 

## Test Plan

`cargo test`

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-03-21 15:48:27 +00:00
Charlie Marsh 46967723bb
Move lowered requirement source type out of `uv-pypi-types` (#12356)
## Summary

This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
2025-03-20 21:16:12 -04:00
Aria Desires 26cad18635
add `default-groups = "all"` sugar to `pyproject.toml` (#12289)
Suggested by @zanieb in #10934

* [x] agree we want to do this
* [x] add docs

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-03-18 13:42:06 -05:00
Charlie Marsh 72be5ffb25
Always reinstall local source trees passed to `uv pip install` (#12176)
## Summary

This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)

Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.

Closes #12038.
2025-03-17 21:12:21 +00:00
John Mumm a5fa7fa996
Replace outdated DevGroups* naming (#11921)
At certain points in the code, dependency groups are represented by
`DevGroups*` naming, probably as a historical artifact. This PR updates
the naming.

This includes renaming `uv-configuration/src/dev.rs` to
`uv-configuration/src/dependency_groups.rs`.
2025-03-03 16:39:44 +01:00
konsti d712ff243e
Edition 2024 prep: Escape `r#gen` and remove redundant ref (#11922)
Three edition 2021 compatible sets of changes in preparation for the
edition 2025 split out from #11724.

In edition 2025, `gen` is a keyword, so we escape it as `r#gen`. `ref`
and `ref mut` are not allowed anymore for `&T` and `&mut T`, so we
remove them. `cargo fmt` now formats inside of macros, which the 2021
formatter doesn't undo.
2025-03-03 11:13:56 +00:00
Charlie Marsh c37af945b3
Avoid using owned `String` in deserializers (#11764)
## Summary

This is the pattern I see in a variety of crates, and I believe this is
preferred if you don't _need_ an owned `String`, since you can avoid the
allocation. This could be pretty impactful for us?
2025-02-25 14:28:16 +00:00
Charlie Marsh 76c3caf24f
Avoid using owned `String` for package name constructors (#11768)
## Summary

Since we use `SmallString` internally, there's no benefit to passing an
owned string to the `PackageName` constructor (same goes for
`ExtraName`, etc.). I've kept them for now (maybe that will change in
the future, so it's useful to have clients passed own values if they
_can_), but removed a bunch of usages where we were casting from `&str`
to `String` needlessly to use the constructor.
2025-02-24 23:06:15 -08:00
Charlie Marsh 46a03b5518
Use a `DryRun` enum everywhere (#11303)
## Summary

Rather than Yet Another Bool.
2025-02-07 00:42:49 +00:00
Aria Desires 72d9361ce1
fix handling of `--all-groups` and `--no-default-groups` flags (#11224)
This is a rewrite of the groups subsystem to have more clear semantics,
and some adjustments to the CLI flag constraints. In doing so, the
following bugs are fixed:

* `--no-default-groups --no-group foo` is no longer needlessly rejected
* `--all-groups --no-default-groups` now correctly evaluates to
`--all-groups` where previously it was erroneously being interpretted as
just `--no-default-groups`
* `--all-groups --only-dev` is now illegal, where previously it was
accepted and mishandled, as if it was a mythical `--only-all-groups`
flag

Fixes #10890
Closes #10891
2025-02-05 15:31:23 -05:00
Matthieu Ancellin 241561979f
Fix typo "dependency-group" in error message and comments (#11211)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

I got a bit confused when testing `[dependency-groups]` because uv's
error message had the same typo I did in my `pyproject.toml`.
I tried to fix it, as well as a few comment I found along the way.
2025-02-04 12:16:05 +01:00
Zanie Blue dd7cd2e86a
Remove warnings for missing lower bounds (#11195)
These are noisy relative to the effect they have on the user. It seems
better to prioritize hints on poor resolutions. Notably, it seems hard
to make these "not noisy" ref #11091.

Does not include the "lowest" resolution mode, in which lower bounds are
critical.
2025-02-03 16:03:31 -06:00
Aria Desires 53706a1864
Add dependency-group cli flags to `uv pip install` and `uv pip compile` (`--group`, `--no-group`, `--only-group`, `--all-groups`) (#10861)
Ultimately this is a lot of settings plumbing and a couple minor pieces
of Actual Logic (which are so simple I have to assume there's something
missing, but maybe not!).

Note this "needlessly" use DevDependencyGroup since it costs nothing, is
more futureproof, and lets us maintain one primary interface (we just
pass `false` for all the dev arguments).

Fixes #8590
Fixes #8969
2025-01-23 08:47:52 -05:00
Mathieu Kniewallner 1479f52be7
feat: add `--no-default-groups` flag (#10618)
## Summary

Closes #10592.

## Test Plan

Snapshot tests.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-21 18:03:17 -05:00
samypr100 4d3809cc6b
Upgrade Rust toolchain to 1.84.0 (#10533)
## Summary
Upgrade the rust toolchain to 1.84.0. This PR does not bump the MSRV.
2025-01-11 22:19:33 -05:00
Charlie Marsh e22b728e3f
Respect PEP 723 script lockfiles in uv run (#10136)
## Summary

If a script has a lockfile, then `uv run` will now reuse it.

Closes https://github.com/astral-sh/uv/issues/7483.

Closes https://github.com/astral-sh/uv/issues/9688.
2025-01-08 20:34:45 +00:00
konsti 3cbf8ab7b1
Optimize `requirements_for_extra` (#10348) 2025-01-07 14:12:12 +00:00
Charlie Marsh 0d57d298e6
Avoid downgrading packages when `--upgrade` is provided (#10097)
## Summary

When `--upgrade` is provided, we should retain already-installed
packages _if_ they're newer than whatever is available from the
registry.

Closes https://github.com/astral-sh/uv/issues/10089.
2025-01-06 17:41:43 -05:00
Charlie Marsh c77aa5820b
Add a required version setting to uv (#10248)
## Summary

This follows Ruff's design exactly: you can provide a version specifier
(like `>=0.5`), and we'll enforce it at runtime.

Closes https://github.com/astral-sh/uv/issues/8605.
2024-12-31 15:37:46 +00:00
Charlie Marsh 4b5a89dbff
Add manylinux2014 aliases for `--python-platform` (#10217)
## Summary

Closes https://github.com/astral-sh/uv/issues/10210.
2024-12-29 01:36:02 +00:00
Charlie Marsh bec8468183
Remove unnecessary prefixes (#10158) 2024-12-25 14:18:01 -05:00
Zanie Blue ae25c2f4db
Upgrade minimum Rust version to 1.83 (#9815)
This reverts commit 6cc7a560f7 to reapply
#9511 since we've disabled ppc64le-musl per #9793
2024-12-11 10:06:19 -06:00
konsti dc82a84841
Build backend: Add template to uv init (#9661)
Add a preview option `uv init --build-backend uv --preview` that uses
the uv build backend when generating the project. The uv build backend
is in preview, so the option is also guarded by preview and hidden from
the help message and docs.

For https://github.com/astral-sh/uv/issues/3957#issuecomment-2518757563
2024-12-05 15:30:48 +01:00
Zanie Blue 6cc7a560f7
Revert "Upgrade to Rust 1.83 (#9511)" (#9617)
This reverts commit cf20673197 (#9511) due
to failure on powerpc64le in https://github.com/astral-sh/uv/pull/9612
2024-12-03 19:21:05 +00:00
Charlie Marsh 8126a5ed32
Make `MarkerTree` `Copy` (#9542)
## Summary

It's just a `usize`. It seems simpler and perhaps even more performant
(?) to make it `Copy`.
2024-11-30 14:07:07 -05:00
Charlie Marsh 57900f33bd
Override `manylinux_compatible` with `--python-platform` (#9526)
## Summary

Closes https://github.com/astral-sh/uv/issues/9521.
2024-11-29 17:26:24 +00:00
Charlie Marsh cf20673197
Upgrade to Rust 1.83 (#9511)
## Summary

A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.

IIUC, this _does_ require bumping our MSRV.
2024-11-29 12:04:22 -05:00
konsti c94777fc54
Initialize rayon lazily (#9435)
When performing a noop sync, we don't need the rayon threadpool, yet we
pay for its initialization:

![Screenshot from 2024-11-26
08-59-07](https://github.com/user-attachments/assets/d918f50d-b5b7-4bdd-820d-cbe71b633aaa)

Be making the initialization lazy, we avoid that cost:

![Screenshot from 2024-11-26
09-53-08](https://github.com/user-attachments/assets/193baea0-667f-4b9d-9a75-886a86f0f837)

This code runs every time before user code in `uv run`.

This means that before calling rayon, one now needs to call
`LazyLock::force(&RAYON_INITIALIZE);`.

Performance mode (CPU 0 is a perf core):
```
$ taskset -c 0 hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
  Time (mean ± σ):       4.5 ms ±   0.1 ms    [User: 2.7 ms, System: 1.8 ms]
  Range (min … max):     4.4 ms …   6.4 ms    640 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
  Time (mean ± σ):       4.4 ms ±   0.1 ms    [User: 2.7 ms, System: 1.6 ms]
  Range (min … max):     4.3 ms …   5.0 ms    679 runs
 
Summary
  /home/konsti/projects/uv/target/profiling/uv sync ran
    1.03 ± 0.04 times faster than /home/konsti/projects/uv/uv-main sync
```

Power saver mode:
```
$ hyperfine --warmup 5 -N "/home/konsti/projects/uv/uv-main sync" "/home/konsti/projects/uv/target/profiling/uv sync"
Benchmark 1: /home/konsti/projects/uv/uv-main sync
  Time (mean ± σ):      28.1 ms ±   1.2 ms    [User: 15.5 ms, System: 20.3 ms]
  Range (min … max):    25.7 ms …  31.9 ms    102 runs
 
Benchmark 2: /home/konsti/projects/uv/target/profiling/uv sync
  Time (mean ± σ):      24.0 ms ±   1.2 ms    [User: 13.8 ms, System: 9.9 ms]
  Range (min … max):    22.2 ms …  28.2 ms    122 runs
 
Summary
  /home/konsti/projects/uv/target/profiling/uv sync ran
    1.17 ± 0.08 times faster than /home/konsti/projects/uv/uv-main sync
```
2024-11-26 14:58:38 +00:00
Li-Lun Lin e485dfd7f1
feat: add support for `--no-extra` flag and setting (#9387)
<!--  
Thank you for contributing to uv! To help us review effectively, please
ensure that:
- The pull request includes a summary of the change.  
- The title is descriptive and concise.  
- Relevant issues are referenced where applicable.  
-->

## Summary

Resolves #9333  

This pull request introduces support for the `--no-extra` command-line
flag and the corresponding `no-extra` UV setting.

### Behavior
- When `--all-extras` is supplied, the specified extras in `--no-extra`
will be excluded from the installation.
- If `--all-extras` is not supplied, `--no-extra` has no effect and is
safely ignored.

## Test Plan

Since `ExtrasSpecification::from_args` and
`ExtrasSpecification::extra_names` are the most important parts in the
implementation, I added the following tests in the
`uv-configuration/src/extras.rs` module:

- **`test_no_extra_full`**: Verifies behavior when `no_extra` includes
the entire list of extras.
- **`test_no_extra_partial`**: Tests partial exclusion, ensuring only
specified extras are excluded.
- **`test_no_extra_empty`**: Confirms that no extras are excluded if
`no_extra` is empty.
- **`test_no_extra_excessive`**: Ensures the implementation ignores
`no_extra` values that don't match any available extras.
- **`test_no_extra_without_all_extras`**: Validates that `no_extra` has
no effect when `--all-extras` is not supplied.
- **`test_no_extra_without_package_extras`**: Confirms correct behavior
when no extras are available in the package.
- **`test_no_extra_duplicates`**: Verifies that duplicate entries in
`pkg_extras` or `no_extra` do not cause errors.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-24 02:25:09 +00:00
Charlie Marsh 0ae9a8b742
Annotate default groups in conflict error messages (#9368)
## Summary

We now tell the user if a group was enabled by default.

Closes https://github.com/astral-sh/uv/issues/9366.
2024-11-22 21:13:15 +00:00
Zanie Blue 8149e636c3
Add aliases for build backend requests (#9294)
I found it surprising we did not accept names like `hatchling` or
`scikit-build`
2024-11-21 14:51:18 +00:00
Charlie Marsh c6482dd038
Remove dev dependencies in `--all-groups --no-dev` (#9300)
## Summary

Closes https://github.com/astral-sh/uv/issues/9297.
2024-11-20 20:06:05 -05:00
Jo 23cc9b0322
Add `--all-groups` to `uv sync|run|export|tree` (#8892)
## Summary

Closes #8594

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-20 16:07:36 +00:00
Hristo Filaretov f49230471c
Add manylinux target triples up to glibc 2.40 (#9234)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

PR #4965 added `*-manylinux_2_31` as a target triple, and issue #4966
described the need for a more general solution.

In lieu of a general solution, this PR adds further explicit manylinux
target triples for different glibc version up to the one used by the
latest Ubuntu release (glibc 2.40 used in Ubuntu 24.10).

## Test Plan

<!-- How was it tested? -->

Local, manual testing with a Python wheel targeting
`x86_64-manylinux_2_35`.
2024-11-19 14:37:43 -05:00
Zanie Blue b642dd9287
Update `uv tool install --force` to imply `--reinstall-package <name>` (#9074) 2024-11-18 19:41:50 -06:00
Charlie Marsh d08bfee718
Remove separate test files in favor of same-file `mod tests` (#9199)
## Summary

These were moved as part of a broader refactor to create a single
integration test module. That "single integration test module" did
indeed have a big impact on compile times, which is great! But we aren't
seeing any benefit from moving these tests into their own files (despite
the claim in [this blog
post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
I see the same compilation pattern regardless of where the tests are
located). Plus, we don't have many of these, and same-file tests is such
a strong Rust convention.
2024-11-18 20:11:46 +00:00
Shantanu 71d9c45393
Turn `--verify-hashes` on by default (#9170)
Fixes #9164

Using clap's `default_value_t` makes the `flag` function unhappy, so
just set the default when we unwrap. Tested with no flags,
`--verify-hashes`, `--no-verify-hashes` and setting in uv.toml

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-18 01:57:54 +00:00
Charlie Marsh 810b430031 Add `--no-group` support to CLI (#8477)
## Summary

Now that `default-groups` can include more than just `"dev"`, it makes
sense to allow users to remove groups with `--no-group`.
2024-10-25 13:27:37 -05:00
Charlie Marsh 291c4c496d Add support for `default-groups` (#8471)
This PR adds support for `tool.uv.default-groups`, which defaults to
`["dev"]` for backwards-compatibility. These represent the groups we
sync by default.
2024-10-25 13:27:37 -05:00
Charlie Marsh c7ccf88939 Error when --group includes non-existent groups (#8394)
## Summary

Part of https://github.com/astral-sh/uv/pull/8272.
2024-10-25 13:27:37 -05:00
Zanie Blue 39ca57f3c8 Add `--group` and `--only-group` to `uv run` (#8274)
Similar to #8110

Part of #8090
2024-10-25 13:27:37 -05:00
Zanie Blue d2e1f180ef Refactor development dependency configuration (#8309)
Part of #8090
Unblocks https://github.com/astral-sh/uv/pull/8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
2024-10-25 13:27:37 -05:00
Zanie Blue 2dc4fd0a19 Add `--group` and `--only-group` to `uv sync` and includes all groups in `uv lock` (#8110)
Part of #8090

Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.

There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
2024-10-25 13:27:37 -05:00
Zanie Blue 4c0590ff6f Add `--group` support to `uv add` and `uv remove` (#8108)
Part of #8090

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
2024-10-25 13:27:37 -05:00
samypr100 319c0183c6
Add templates for popular build backends (#7857)
Co-authored-by: konstin <konstin@mailbox.org>
2024-10-16 14:19:59 +02:00
Charlie Marsh 59003cb021
Avoid showing lower-bound warning outside of explicit lock and sync (#8234)
## Summary

We shouldn't show these in `uv add`, especially when the thing we're
adding is about to have a lower-bound put on it. Now, we only show these
when the user runs `uv lock` or `uv sync`.
2024-10-15 20:49:40 -04:00
samypr100 01c44af3c3
chore: unify all env vars used (#8151)
## Summary

This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.

I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.

This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.

I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.

I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.

Did I miss any? Are their initial docs cohesive?

Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.

## Test Plan

Existing tests
2024-10-14 16:48:13 -05:00
Amos Wenger a3b11dacb8
Support for wildcard in UV_INSECURE_HOST (#8052)
Allow '*' as a value to match all hosts, and provide
`reqwest_blocking_get` for uv tests, so that they also respect
UV_INSECURE_HOST (since they respect `ALL_PROXY`).

This lets those tests pass with a forward proxy - we can think about
setting a root certificate later so that we don't need to disable
certificate verification at all.

---

I tested this locally by running:

```bash
GIT_SSL_NO_VERIFY=true ALL_PROXY=localhost:8080 UV_INSECURE_HOST="*" cargo nextest run sync_wheel_path_source_error
```

With my forward proxy showing:

```
2024-10-09T18:20:16.300188Z  INFO fopro: Proxied GET cc2fedbd88a6546c1727ae13fa977a/cffi-1.17.1-cp310-cp310-macosx_11_0_arm64.whl (headers 480.024958ms + body 92.345666ms)
2024-10-09T18:20:16.913298Z  INFO fopro: Proxied GET https://pypi.org/simple/pycparser/ (headers 509.664834ms + body 269.291µs)
2024-10-09T18:20:17.383975Z  INFO fopro: Proxied GET 5f610ebe4298517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl.metadata (headers 443.184208ms + body 2.094792ms)
```
2024-10-12 13:55:26 +01:00
Amos Wenger 715f28fd39
chore: Move all integration tests to a single binary (#8093)
As per
https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html

Before that, there were 91 separate integration tests binary.

(As discussed on Discord — I've done the `uv` crate, there's still a few
more commits coming before this is mergeable, and I want to see how it
performs in CI and locally).
2024-10-11 16:41:35 +02:00
Charlie Marsh 14507a1793
Add `uv-` prefix to all internal crates (#7853)
## Summary

Brings more consistency to the repo and ensures that all crates
automatically show up in `--verbose` logging.
2024-10-01 20:15:32 -04:00
Charlie Marsh b6de417c94
Use `serde-untagged` to improve some untagged enum error messages (#7822)
## Summary

This is related to https://github.com/astral-sh/uv/issues/7817, but
doesn't close it.
2024-09-30 23:40:21 +00:00
Jo 0c801f8f4b
Initialize a Git repository in `uv init` (#5476)
## Summary

Similiar to `cargo init --vcs <VCS>`, this PR adds the `--vcs <VCS>`
flag for `uv init`, allowing to create a version control system during
initialization. By default, `uv init` will create a Git repository if
the `--vcs` flag is not provided. Use `--vcs none` to disable this
feature.

Currently, only Git is supported. While Cargo also supports hg, pijul,
and fossil, this initial PR only includes Git. We can add more later if
there are any user requests.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-09-26 02:40:39 +00:00
Zanie Blue 12ab7d1ab7
Respect `--quiet` flag in `uv build` (#7674)
I noticed this was not respected, seems like an oversight.
2024-09-25 08:23:43 -05:00
konsti 205bf8cabe
Implement trusted publishing (#7548)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-09-24 16:07:20 +00:00
Charlie Marsh 778da3350a
Add `--no-editable` support to `uv sync` and `uv export` (#7371)
## Summary

Closes https://github.com/astral-sh/uv/issues/5792.
2024-09-17 14:50:36 +00:00
Charlie Marsh 5f2e536925
Add support for `--only-dev` to `uv sync` and `uv export` (#7367)
## Summary

Closes https://github.com/astral-sh/uv/issues/7255.

Closes https://github.com/astral-sh/uv/issues/6472.
2024-09-16 20:06:20 +00:00
Charlie Marsh 15792a3775
Apply `--no-install` options when constructing resolution (#7277)
## Summary

We need to apply the `--no-install` filters earlier, such that we don't
error if we only have a source distribution for a given package when
`--no-build` is provided but that package is _omitted_.

Closes #7247.
2024-09-11 14:31:24 -04:00
Charlie Marsh 95a4beeed3
Remove workspace root for single-member workspace with `uv export` (#7254)
## Summary

If we have a single-member workspace, we actually don't write it to
`members`.

Closes https://github.com/astral-sh/uv/issues/7241.
2024-09-10 09:27:45 -04:00
Charlie Marsh 4f2349119c
Add support for dynamic cache keys (#7136)
## Summary

This PR adds a more flexible cache invalidation abstraction for uv, and
uses that new abstraction to improve support for dynamic metadata.

Specifically, instead of relying solely on a timestamp, we now pass
around a `CacheInfo` struct which (as of now) contains
`Option<Timestamp>` and `Option<Commit>`. The `CacheInfo` is saved in
`dist-info` as `uv_cache.json`, so we can test already-installed
distributions for cache validity (along with testing _cached_
distributions for cache validity).

Beyond the defaults (`pyproject.toml`, `setup.py`, and `setup.cfg`
changes), users can also specify additional cache keys, and it's easy
for us to extend support in the future. Right now, cache keys can either
be instructions to include the current commit (for `setuptools_scm` and
similar) or file paths (for `hatch-requirements-txt` and similar):

```toml
[tool.uv]
cache-keys = [{ file = "requirements.txt" }, { git = true }]
```

This change should be fully backwards compatible.

Closes https://github.com/astral-sh/uv/issues/6964.

Closes https://github.com/astral-sh/uv/issues/6255.

Closes https://github.com/astral-sh/uv/issues/6860.
2024-09-09 20:19:15 +00:00
Charlie Marsh 6ae005b0d0
Add `--no-emit-project` and friends to `uv export` (#7110)
## Summary

Like `uv sync`, you can omit the current project (`--no-emit-project`),
a specific package (`--no-emit-package`), or the entire workspace
(`--no-emit-workspace`).

Closes https://github.com/astral-sh/uv/issues/6960.

Closes #6995.
2024-09-06 01:01:46 +00:00
Charlie Marsh 05ed4bc11d
Show build output by default in `uv build` (#6912)
## Summary

This is a big improvement IMO:

![Screenshot 2024-09-01 at 12 52
28 PM](https://github.com/user-attachments/assets/2d8b1370-3385-429a-9a1d-e1d44611a2b4)
2024-09-04 15:39:21 +00:00
Charlie Marsh 42a4d80a63
Add source distribution support to `uv-build` crate (#6896)
## Summary

Just exposes the correct PEP 517 hooks.
2024-09-02 18:14:49 +00:00
Charlie Marsh cbfc928a9c
Add `uv export --format requirements.txt` (#6778)
## Summary

The interface here is intentionally a bit more limited than `uv pip
compile`, because we don't want `requirements.txt` to be a system of
record -- it's just an export format. So, we don't write annotation
comments (i.e., which dependency is requested from which), we don't
allow writing extras, etc. It's just a flat list of requirements, with
their markers and hashes.

Closes #6007.

Closes #6668.

Closes #6670.
2024-08-29 17:46:42 +00:00
Charlie Marsh cef3d35405
Support `{package}@{version}` in `uv tool install` (#6762)
## Summary

Closes https://github.com/astral-sh/uv/issues/6759.

Closes https://github.com/astral-sh/uv/issues/6535.
2024-08-28 12:40:49 -04:00
Charlie Marsh af323888ee
Accept either strings or structs for hosts (#6763)
## Summary

Technically a struct did work in the last release, so let's not break
it.
2024-08-28 16:36:12 +00:00
Charlie Marsh 53ef633c6d
Do not require workspace members to sync with `--frozen` (#6737)
## Summary

Closes https://github.com/astral-sh/uv/issues/6685.
2024-08-28 07:58:50 -04:00
Charlie Marsh ffb0c304d1
Implement deserialization for trusted host (#6716) 2024-08-27 19:30:44 +00:00
Charlie Marsh d86075fc1e
Add support for `--trusted-host` (#6591)
## Summary

This PR revives https://github.com/astral-sh/uv/pull/4944, which I think
was a good start towards adding `--trusted-host`. Last night, I tried to
add `--trusted-host` with a custom verifier, but we had to vendor a lot
of `reqwest` code and I eventually hit some private APIs. I'm not
confident that I can implement it correctly with that mechanism, and
since this is security, correctness is the priority.

So, instead, we now use two clients and multiplex between them.

Closes https://github.com/astral-sh/uv/issues/1339.

## Test Plan

Created self-signed certificate, and ran `python3 -m http.server --bind
127.0.0.1 4443 --directory . --certfile cert.pem --keyfile key.pem` from
the packse index directory.

Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
https://127.0.0.1:8443/simple-html` failed with:

```
error: Request failed after 3 retries
  Caused by: error sending request for url (https://127.0.0.1:8443/simple-html/transitive-yanked-and-unyanked-dependency-a-0abad3b6/)
  Caused by: client error (Connect)
  Caused by: invalid peer certificate: Other(OtherError(CaUsedAsEndEntity))
```

Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.1:8443'`
failed with the expected error (invalid resolution) and made valid
requests.

Verified that `cargo run pip install
transitive-yanked-and-unyanked-dependency-a-0abad3b6 --index-url
'https://127.0.0.1:8443/simple-html' --trusted-host '127.0.0.2' -n` also
failed.
2024-08-27 09:36:50 -04:00
Mathieu Kniewallner 6a988aca55
refactor: use a struct for install options (#6561)
## Summary

Closes #6545.

## Test Plan

Relying on existing tests.
2024-08-27 05:38:16 -05:00
Charlie Marsh 732d2fb0fb Remove `--legacy-setup-py` command-line argument (#4255)
This is a fallback mode that we supported when we decided to use PEP 517
builds by default. I can't find a single reference to it on GitHub or in
our issue tracker, so I want to drop support for it as part of v0.3.0.
2024-08-20 11:31:46 -05:00
Charlie Marsh 81a50dcb08
Add 32-bit Windows target (#6252)
## Summary

Closes https://github.com/astral-sh/uv/issues/6245.

## Test Plan

```
❯ cargo run pip install sqlalchemy --python-platform i686-pc-windows-msvc --verbose --no-cache --reinstall
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.16s
     Running `target/debug/uv pip install sqlalchemy --python-platform i686-pc-windows-msvc --verbose --no-cache --reinstall`
DEBUG uv 0.2.37
DEBUG Searching for Python interpreter in system path
DEBUG Found `cpython-3.12.3-macos-aarch64-none` at `/Users/crmarsh/workspace/uv/.venv/bin/python3` (virtual environment)
DEBUG Using Python 3.12.3 environment at .venv/bin/python3
DEBUG Acquired lock for `.venv`
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.12.3
DEBUG Adding direct dependency: sqlalchemy*
DEBUG No cache entry for: https://pypi.org/simple/sqlalchemy/
WARN Skipping file for sqlalchemy: SQLAlchemy-0.1.1-py2.4.egg
WARN Skipping file for sqlalchemy: SQLAlchemy-0.1.2-py2.4.egg
WARN Skipping file for sqlalchemy: SQLAlchemy-0.1.3-py2.4.egg
WARN Skipping file for sqlalchemy: SQLAlchemy-0.1.4-py2.4.egg
DEBUG Searching for a compatible version of sqlalchemy (*)
DEBUG Selecting: sqlalchemy==2.0.32 [compatible] (SQLAlchemy-2.0.32-cp312-cp312-win32.whl)
DEBUG No cache entry for: 973e0bbf2b36c3c06fd1dc8480c209/SQLAlchemy-2.0.32-cp312-cp312-win32.whl.metadata
DEBUG Adding transitive dependency for sqlalchemy==2.0.32: typing-extensions>=4.6.0
DEBUG No cache entry for: https://pypi.org/simple/typing-extensions/
DEBUG Searching for a compatible version of typing-extensions (>=4.6.0)
DEBUG Selecting: typing-extensions==4.12.2 [compatible] (typing_extensions-4.12.2-py3-none-any.whl)
DEBUG No cache entry for: ad63fc024801216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl.metadata
DEBUG Tried 2 versions: sqlalchemy 1, typing-extensions 1
DEBUG Split specific environment resolution took 0.390s
Resolved 2 packages in 391ms
DEBUG Must revalidate requirement: sqlalchemy
DEBUG Must revalidate requirement: typing-extensions
DEBUG Unnecessary package: markupsafe==2.1.5
DEBUG Unnecessary package: filelock==3.15.4
DEBUG Unnecessary package: fsspec==2024.6.1
DEBUG Unnecessary package: jinja2==3.1.4
DEBUG Unnecessary package: mpmath==1.3.0
DEBUG Unnecessary package: networkx==3.3
DEBUG Unnecessary package: setuptools==72.2.0
DEBUG Unnecessary package: sympy==1.13.2
DEBUG Unnecessary package: torch==2.4.0
DEBUG No cache entry for: 973e0bbf2b36c3c06fd1dc8480c209/SQLAlchemy-2.0.32-cp312-cp312-win32.whl
DEBUG No cache entry for: ad63fc024801216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl
Prepared 2 packages in 150ms
DEBUG Uninstalled sqlalchemy (275 files, 25 directories)
DEBUG Uninstalled typing-extensions (7 files, 1 directory)
```
2024-08-20 14:06:25 +00:00
Ibraheem Ahmed f5110f7b5e
Remove uses of `Option<MarkerTree>` (#5978)
## Summary

Follow up to https://github.com/astral-sh/uv/pull/5898. This should fix
some of the failures in https://github.com/astral-sh/uv/pull/5887 where
`uv lock --locked` is failing due to `Some(true)` and `None` markers not
comparing equal.
2024-08-10 13:23:29 -04:00
Charlie Marsh f89403f4f6
Retain and respect settings in tool upgrades (#5937)
## Summary

We now persist the `ResolverInstallerOptions` when writing out a tool
receipt. When upgrading, we grab the saved options, and merge with the
command-line arguments and user-level filesystem settings (CLI > receipt
> filesystem).
2024-08-09 18:21:49 +00:00
Ibraheem Ahmed ffd18cc75d
Implement marker trees using algebraic decision diagrams (#5898)
## Summary

This PR rewrites the `MarkerTree` type to use algebraic decision
diagrams (ADD). This has many benefits:
- The diagram is canonical for a given marker function. It is impossible
to create two functionally equivalent marker trees that don't refer to
the same underlying ADD. This also means that any trivially true or
unsatisfiable markers are represented by the same constants.
- The diagram can handle complex operations (conjunction/disjunction) in
polynomial time, as well as constant-time negation.
- The diagram can be converted to a simplified DNF form for user-facing
output.

The new representation gives us a lot more confidence in our marker
operations and simplification, which is proving to be very important
(see https://github.com/astral-sh/uv/pull/5733 and
https://github.com/astral-sh/uv/pull/5163).

Unfortunately, it is not easy to split this PR into multiple commits
because it is a large rewrite of the `marker` module. I'd suggest
reading through the `marker/algebra.rs`, `marker/simplify.rs`, and
`marker/tree.rs` files for the new implementation, as well as the
updated snapshots to verify how the new simplification rules work in
practice. However, a few other things were changed:
- [We now use release-only comparisons for `python_full_version`, where
we previously only did for
`python_version`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/algebra.rs#L522).
I'm unsure how marker operations should work in the presence of
pre-release versions if we decide that this is incorrect.
- [Meaningless marker expressions are now
ignored](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/parse.rs#L502).
This means that a marker such as `'x' == 'x'` will always evaluate to
`true` (as if the expression did not exist), whereas we previously
treated this as always `false`. It's negation however, remains `false`.
- [Unsatisfiable markers are written as `python_version <
'0'`](https://github.com/astral-sh/uv/blob/ibraheem/canonical-markers/crates/pep508-rs/src/marker/tree.rs#L1329).
- The `PubGrubSpecifier` type has been moved to the new `uv-pubgrub`
crate, shared by `pep508-rs` and `uv-resolver`. `pep508-rs` also depends
on the `pubgrub` crate for the `Range` type, we probably want to move
`pubgrub::Range` into a separate crate to break this, but I don't think
that should block this PR (cc @konstin).

There is still some remaining work here that I decided to leave for now
for the sake of unblocking some of the related work on the resolver.
- We still use `Option<MarkerTree>` throughout uv, which is unnecessary
now that `MarkerTree::TRUE` is canonical.
- The `MarkerTree` type is now interned globally and can potentially
implement `Copy`. However, it's unclear if we want to add more
information to marker trees that would make it `!Copy`. For example, we
may wish to attach extra and requires-python environment information to
avoid simplifying after construction.
- We don't currently combine `python_full_version` and `python_version`
markers.
- I also have not spent too much time investigating performance and
there is probably some low-hanging fruit. Many of the test cases I did
run actually saw large performance improvements due to the markers being
simplified internally, reducing the stress on the old `normalize`
routine, especially for the extremely large markers seen in
`transformers` and other projects.

Resolves https://github.com/astral-sh/uv/issues/5660,
https://github.com/astral-sh/uv/issues/5179.
2024-08-09 13:40:02 -04:00
Charlie Marsh fd1d508108
Make `--upgrade` imply `--refresh` (#5943)
## Summary

I think this seems reasonable... Otherwise, we might not go back to PyPI
to revalidate the list of available versions despite the user passing
`--upgrade`.
2024-08-08 20:11:31 -04:00
Charlie Marsh 089f50a845
Add `--no-sources` to avoid reading from `tool.uv.sources` (#5801)
## Summary

Closes https://github.com/astral-sh/uv/issues/5791.
2024-08-06 14:14:19 +00:00
Charlie Marsh d0919329fd
Make `--reinstall` imply `--refresh` (#5425)
## Summary

It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.

This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.

Closes https://github.com/astral-sh/uv/issues/5424.
2024-07-25 09:45:58 -04:00
Charlie Marsh 5a23f05799
Store resolution options in lockfile (#5264)
## Summary

This PR modifies the lockfile to include the impactful resolution
settings, like the resolution and pre-release mode. If any of those
values change, we want to ignore the existing lockfile. Otherwise,
`--resolution lowest-direct` will typically have no effect, which is
really unintuitive.

Closes https://github.com/astral-sh/uv/issues/5226.
2024-07-22 08:28:22 -04:00
Charlie Marsh 82d94838cb
Implement a `--verify-hashes` hash-checking mode (#4007)
## Summary

This is an alternative to `--require-hashes` which will validate a hash
if it's present, but ignore requirements that omit hashes or are absent
from the lockfile entirely.

So, e.g., transitive dependencies that are missing will _not_ error; nor
will dependencies that are included but lack a hash.

Closes https://github.com/astral-sh/uv/issues/3305.
2024-07-17 21:25:31 +00:00
Charlie Marsh 7925d255f7
Add `manylinux_2_31` to supported `--python-platform` (#4965)
## Summary

I'll make an issue for arbitrary `manylinux_x_y` support.

Closes https://github.com/astral-sh/uv/issues/4956.
2024-07-10 16:30:35 +00:00
Charlie Marsh 23eb42deed
Allow constraints to be provided in `--upgrade-package` (#4952)
## Summary

Allows, e.g., `--upgrade-package flask<3.0.0`.

Closes https://github.com/astral-sh/uv/issues/1964.
2024-07-09 20:09:13 -07:00
konsti 53db63f6dd
Apply extra to overrides and constraints (#4829)
This is an attempt to solve https://github.com/astral-sh/uv/issues/ by
applying the extra marker of the requirement to overrides and
constraints.

Say in `a` we have a requirements
```
b==1; python_version < "3.10"
c==1; extra == "feature"
```

and overrides
```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10"
c==3; python_version >= "3.10"
```

Our current strategy is to discard the markers in the original
requirements. This means that on 3.12 for `a` we install `b==3`, but it
also means that we add `c` to `a` without `a[feature]`, causing #4826.
With this PR, the new requirement become,

```
b==2; python_version < "3.10"
b==3; python_version >= "3.10"
c==2; python_version < "3.10" and extra == "feature"
c==3; python_version >= "3.10" and extra == "feature"
```

allowing to override markers while preserving optional dependencies as
such.

Fixes #4826
2024-07-09 20:37:24 +02:00
konsti f2f48d339e
Flatten requirements eagerly in `get_dependencies` (#4430)
Downstack PR: #4515 Upstack PR: #4481

Consider these two cases:

A:
```
werkzeug==2.0.0
werkzeug @ 960bb4017c4aed12b5ed8b78e0153e/Werkzeug-2.0.0-py3-none-any.whl
```

B:
```toml
dependencies = [
  "iniconfig == 1.1.1 ; python_version < '3.12'",
  "iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```

In the first case, `werkzeug==2.0.0` should be overridden by the url. In
the second case `iniconfig == 1.1.1` is in a different fork and must
remain a registry distribution.

That means the conversion from `Requirement` to `PubGrubPackage` is
dependent on the other requirements of the package. We can either look
into the other packages immediately, or we can move the forking before
the conversion to `PubGrubDependencies` instead of after. Either version
requires a flat list of `Requirement`s to use. This refactoring gives us
this list.

I'll add support for both of the above cases in the forking urls branch
before merging this PR. I also have to move constraints over to this.
2024-06-25 21:13:47 +00:00
Charlie Marsh f07308823e
Add `--emit-build-options` flag to `uv pip compile` interface (#4463)
## Summary

Closes https://github.com/astral-sh/uv/issues/4420.
2024-06-24 12:25:01 +00:00
renovate[bot] 3251690327
Update Rust crate rustc-hash to v2 (#4461) 2024-06-24 01:13:05 +00:00
Charlie Marsh 7d9541d0f4
Add `--no-build`, `--no-build-package`, and binary variants (#4322)
## Summary

These are now supported on `uv run`, `uv lock`, `uv sync`, and `uv tool
run`.

Closes https://github.com/astral-sh/uv/issues/4297.
2024-06-14 04:05:00 +00:00
Zanie Blue 5a007b6b9f
Add `BuildOptions` for centralized combination of `NoBuild` and `NoBinary` (#4284)
As requested in review of https://github.com/astral-sh/uv/pull/4067
2024-06-12 21:33:33 +00:00
konsti 081f20c53e
Add support for `tool.uv` into distribution building (#3904)
With the change, we remove the special casing of workspace dependencies
and resolve `tool.uv` for all git and directory distributions. This
gives us support for non-editable workspace dependencies and path
dependencies in other workspaces. It removes a lot of special casing
around workspaces. These changes are the groundwork for supporting
`tool.uv` with dynamic metadata.

The basis for this change is moving `Requirement` from
`distribution-types` to `pypi-types` and the lowering logic from
`uv-requirements` to `uv-distribution`. This changes should be split out
in separate PRs.

I've included an example workspace `albatross-root-workspace2` where
`bird-feeder` depends on `a` from another workspace `ab`. There's a
bunch of failing tests and regressed error messages that still need
fixing. It does fix the audited package count for the workspace tests.
2024-05-31 02:42:03 +00:00
Charlie Marsh 3461c8b585
Move extra specification into `uv-configuration` (#3897)
## Summary

I need to use this in the resolver (and it's at-home with other, similar
configuration options).
2024-05-29 04:49:57 +00:00
Charlie Marsh 73f67089e1
Add JSON alias for `unsafe-any-match` (#3820)
Closes https://github.com/astral-sh/uv/issues/3819.
2024-05-24 13:31:46 +00:00
Charlie Marsh 8d566e553d
Use a cross-platform representation for relative paths in `pip compile` (#3804)
## Summary

I haven't tested on Windows yet, but the idea here is that we should use
a portable representation when printing paths.

I decided to limit the scope here to paths that we write to output
files.

Closes https://github.com/astral-sh/uv/issues/3800.
2024-05-24 03:02:42 +00:00
Charlie Marsh dd27f2f710
Make anyhow a dev dependency in `uv-configuration` (#3814) 2024-05-24 02:49:31 +00:00