Commit Graph

7187 Commits

Author SHA1 Message Date
Andrew Gallant 7d2abd0549 clippy: allow `if_not_else`
This lint fires when an `if` negates the condition. The lint
instead suggests that the condition should be un-negated and
the `if` and `else` bodies flipped.

I find this to be a pretty annoying lint, because sometimes
I want to keep the order of the `if` and `else` bodies as-is,
and not make it subject to whether the primary condition is
negated or not. Sometimes it's for linear scanning reasons
(where the `if` block is smaller), and sometimes it's for
"code parallelism" reasons (i.e., this block of code looks like
another block of code intentionally).

Moreover, I don't think the benefits of this lint are very
big. I do agree that sometimes negating a conditional can make
it harder to read (e.g., a double negative), but it's nowhere
near a universal truth enough to ban it outright.
2024-12-02 09:09:35 -05:00
Alex Willmer 8d01f70beb
Add `--dry-run` to `uv pip uninstall` (#9557)
## Summary

This proposes adding the command line option `uv pip uninstall --dry-run
...`, complementing the existing `uv pip install --dry-run ...` added
for #1244 in #1436.

This option does not exist in PyPA's `pip uninstall`, if adopted it
would be unique to `uv pip`. The code should be considered PoC, it is
baby's first Rust.

The initial motivation was while investigating
https://github.com/moreati/ansible-uv/issues/2 - to allow Ansible module
`moreati.uv.pip` to work with`state: absent` in "check_mode" (Ansible's
equivalent of a dry run), without requiring `packaging` or `setuptools`.

## Test Plan

One new unit test has been added. I pedge to add more if the feature is
desired/accepted

Example usage

```console
➜  uv git:(pip-uninstall--dry-run) rm -rf .venv
➜  uv git:(pip-uninstall--dry-run) ./target/debug/uv venv                   
Using CPython 3.13.0
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
➜  uv git:(pip-uninstall--dry-run) ./target/debug/uv pip install httpx      
Resolved 7 packages in 178ms
Prepared 5 packages in 60ms
Installed 7 packages in 15ms
 + anyio==4.6.2.post1
 + certifi==2024.8.30
 + h11==0.14.0
 + httpcore==1.0.7
 + httpx==0.28.0
 + idna==3.10
 + sniffio==1.3.1
➜  uv git:(pip-uninstall--dry-run) ./target/debug/uv pip uninstall --dry-run httpx
Would uninstall 1 package
 - httpx==0.28.0
➜  uv git:(pip-uninstall--dry-run) ./target/debug/uv pip list                     
Package  Version
-------- -----------
anyio    4.6.2.post1
certifi  2024.8.30
h11      0.14.0
httpcore 1.0.7
httpx    0.28.0
idna     3.10
sniffio  1.3.1
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-12-02 02:57:47 +00:00
renovate[bot] 89a25ba465
Update Rust crate rustc-hash to v2.1.0 (#9569) 2024-12-02 01:23:42 +00:00
renovate[bot] b5807754fe
Update Rust crate indexmap to v2.7.0 (#9567) 2024-12-01 20:07:06 -05:00
renovate[bot] 2ee710a620
Update tokio-tracing monorepo (#9566) 2024-12-01 20:06:59 -05:00
renovate[bot] 93d86f582f
Update pre-commit dependencies (#9565) 2024-12-01 20:06:53 -05:00
renovate[bot] 0a42c3c340
Update Rust crate syn to v2.0.90 (#9564) 2024-12-01 20:06:47 -05:00
renovate[bot] 3ee6ef2556
Update Rust crate rkyv to v0.8.9 (#9563) 2024-12-01 20:06:40 -05:00
renovate[bot] f92640cf2a
Update Rust crate pathdiff to v0.2.3 (#9562) 2024-12-01 20:06:34 -05:00
renovate[bot] fb64fb7e13
Update Rust crate jiff to v0.1.15 (#9561) 2024-12-01 20:06:28 -05:00
renovate[bot] ef20d6a92a
Update Rust crate cargo-util to v0.2.16 (#9560) 2024-12-01 20:06:20 -05:00
Jp 03f8808d11
Handle Windows AV/EDR file locks during script installations (#9543)
Fixes #9531


## Context

While working with [uv](https://github.com/astral-sh/uv), I encountered
issues with a python dependency, [httpx](https://www.python-httpx.org/)
unable to be installed because of a **os error 5 permission denied**.
The error occur when we try to persist a **.exe file** from a temporary
folder into a persistent one.
I only reproduce the issue in an enterprise **Windows** Jenkins Runner.
In my virtual machines, I don't have any issues. So I think this is most
probably coming from the system configuration. This windows runner
**contains an AV/EDR**. And the fact that the file locked occured only
once for an executable make me think that it's most probably the cause.

While doing some research and speaking with some colleagues (hi
@vmeurisse), it seems that the issue is a very recurrent one on Windows.
In the Javascript ecosystem, there is this package, created by the
@isaacs, `npm` inventor: https://www.npmjs.com/package/graceful-fs, used
inside `npm`, allowing its package installations to be more resilient to
filesystem errors:
> The improvements are meant to normalize behavior across different
platforms and environments, and to make filesystem access more resilient
to errors.
One of its core feature is this one:
> On Windows, it retries renaming a file for up to one second if EACCESS
or EPERM error occurs, likely because antivirus software has locked the
directory.

So I tried to implement the same algorithm on `uv`, **and it fixed my
issue**! I can finally install `httpx`.

Then, [as I mentionned in this
issue](https://github.com/astral-sh/uv/issues/9531#issuecomment-2508981316),
I saw that you already implemented exactly the same algorithm in an
asynchronous function for renames 😄

22fd9f7ff1/crates/uv-fs/src/lib.rs (L221)

## Summary of changes 
- I added a similar function for `persist` (was not easy to have the
benediction of the borrow checker 😄)
- I added a `sync` variant of `rename_with_retry`
- I edited `install_script` to use the function including retries on
Windows

Let me know if I should change anything 🙂 

Thanks!!

## Test Plan
This pull-request should be totally iso-functional, so I think it should
be covered by existing tests in case of regression.
All tests are still passing on my side.
Also, of course validated that my windows machines (windows 10 & windows
11) containing AV/EDR software are now able to install `httpx.exe`
script.
However, if you think any additional test is needed, feel free to tell
me!
2024-12-01 17:57:09 -05:00
konsti 8a863194c8
Use `SharedState` for build dispatch (#9553)
When looking at the build frontend code, I noticed that we always pass
every single field of the shared state to the build dispatch:

```rust
    let build_dispatch = BuildDispatch::new(
        ...
        &state.index,
        &state.git,
        &state.capabilities,
        &state.in_flight,
        ...
    );
```

We can abstract this by moving `SharedState` into the build dispatch.
The `BuildDispatch` then has only immutable fields and the
`SharedState`. Since the `SharedState` is all `Arc`s, we can clone it
freely.
2024-12-01 17:20:28 -05:00
Jp 99abd6854e
Align `indoc` and `base64` workspace dependencies with root project (#9555)
## Summary
After #9524, I noticed two other dependencies were misaligned.
Since the previous PR has been merged, I was thinking I could submit
those two misses.
Of course, open to any comments/decline!
Thanks!! 🙂 

## Test Plan
All units tests are still passing on my side. Let's see with the
pull-request CI again 😄
2024-12-01 17:20:22 -05:00
Charlie Marsh 71df509d09
Avoid adding non-extra package with extra dependencies (#9540)
## Summary

Previously, when we encountered `foo[bar]`, we'd add a dependency on
`PubGrubPackage::Package` for `foo`, and then `PubGrubPackage::Extra`
for `foo[bar]`.

Later, when we ask for the dependencies of the `PubGrubPackage::Extra`,
we add `PubGrubPackage::Package` for `foo`, and
`PubGrubPackage::Package` for `foo[bar]`. This is an intentional
strategy because it ensures that PubGrub "knows" that these have to be
solved to the same version as early as possible.

It turns out that the first part here ("add a dependency on
`PubGrubPackage::Package` for `foo`") is suboptimal, because it means
PubGrub might try to solve _just_ `foo` without realizing that it also
has to accommodate all the constraints from the extra.

Instead, we now add _just_ `PubGrubPackage::Extra` for `foo[bar]`, and
defer adding the base package. It looks like this leads to a far more
efficient solve for Airflow.
2024-12-01 08:42:27 -05:00
konsti bb70382dac
Build backend: Default excludes (#9552)
When adding excludes, we usually don't want to include python cache
files. On the contrary, I haven't seen any project in my ecosystem
research that would want any of `__pycache__`, `*.pyc`, `*.pyo` to be
included. By moving them behind a `default-excludes` toggle, they are
always active even when defining custom excludes, but can be deactivated
if the user so chooses.

With includes and excludes being this small again, we can roll back the
include-exclude anchored difference to always using anchored globs (i.e.
you would need to use `**/build-*.h` below).

A pyproject.toml with custom settings with the change applied:

```toml
[project]
name = "foo"
version = "0.1.0"
readme = "README.md"
license-files = ["LICENSE*", "third-party-licenses/*"]

[tool.uv.build-backend]
# A file we need for the source dist -> wheel step, but not in the wheel itself (currently unused)
source-include = ["data/build-script.py"]
# A temporary or generated file we want to ignore
source-exclude = ["/src/foo/not-packaged.txt"]
# Headers are build-only
wheel-exclude = ["build-*.h"]

[tool.uv.build-backend.data]
scripts = "scripts"
data = "assets"
headers = "header"

[build-system]
requires = ["uv>=0.5.5,<0.6"]
build-backend = "uv"
```
2024-12-01 14:09:55 +01:00
konsti dfcceb6a1d
Build backend: Revamp include/exclude (#9525)
When building the source distribution, we always need to include
`pyproject.toml` and the module, when building the wheel, we always
include the module but nothing else at top level. Since we only allow a
single module per wheel, that means that there are no specific wheel
includes. This means we have source includes, source excludes, wheel
excludes, but no wheel includes: This is defined by the module root,
plus the metadata files and data directories separately.

Extra source dist includes are currently unused (they can't end up in
the wheel currently), but it makes sense to model them here, they will
be needed for any sort of procedural build step.

This results in the following fields being relevant for inclusions and
exclusion:

* `pyproject.toml` (always included in the source dist)
* project.readme: PEP 621
* project.license-files: PEP 639
* module_root: `Path`
* source_include: `Vec<Glob>`
* source_exclude: `Vec<Glob>`
* wheel_exclude: `Vec<Glob>`
* data: `Map<KnownDataName, Path>`

An opinionated choice is that that wheel excludes always contain the
source excludes: Otherwise you could have a path A in the source tree
that gets included when building the wheel directly from the source
tree, but not when going through the source dist as intermediary,
because A is in source excludes, but not in the wheel excludes. This has
been a source of errors previously.

In the process, I fixed a bug where we would skip directories and only
include the files and were missing license due to absolute globs.
2024-12-01 11:32:35 +00:00
Charlie Marsh 8126a5ed32
Make `MarkerTree` `Copy` (#9542)
## Summary

It's just a `usize`. It seems simpler and perhaps even more performant
(?) to make it `Copy`.
2024-11-30 14:07:07 -05:00
Charlie Marsh 6bebf79ac3
Remove uses of `Option<MarkerTreeContents>` in `PubGrubPackage` (#9541)
## Summary

Just use `MarkerTree::TRUE` instead of `None`.
2024-11-30 10:36:18 -05:00
Charlie Marsh 22fd9f7ff1
Pass extra when evaluating lockfile markers (#9539)
## Summary

When we serialize and deserialize the lockfile, we remove the conflict
markers. So in the linked case, the edges for the `tqdm` entries are
like:

```
complexified_marker: UniversalMarker {
    pep508_marker: python_full_version >= '3.9.0',
    conflict_marker: true,
},
```

However... when we evaluate in-memory, the conflict markers are still
there...

```
complexified_marker: UniversalMarker {
    pep508_marker: true,
    conflict_marker: extra == 't1' and extra != 't2',
},
```

So if `uv run` creates the lockfile, we evaluate this as `false`.

We should make this consistent, and I expect @BurntSushi is aware. But
for now, it's reasonable / correct to pass the extra when evaluating at
this specific point, since we know the dependency was enabled by the
marker.

Closes
https://github.com/astral-sh/uv/issues/9533#issuecomment-2508908591.
2024-11-30 14:22:18 +00:00
Charlie Marsh 53fe301b1d
Avoid using IDs across PubGrub states (#9538)
## Summary

This isn't safe, because the prefetcher is global but the IDs could come
from different PubGrub states (i.e., different forks).
2024-11-30 13:59:21 +00:00
Charlie Marsh 2aca623691
Remove unused specifier field from `PubGrub` structs (#9536) 2024-11-30 03:59:34 +00:00
Charlie Marsh 69811837e5
Support recursive extras with marker in `pip compile -r pyproject.toml` (#9535)
## Summary

Closes https://github.com/astral-sh/uv/issues/9530.
2024-11-30 03:40:22 +00:00
Nathan McDougall 891e02d586
Add missing word to docs for run.md (#9527)
## Summary

I found this sentence didn't read quite right without this "the".

## Test Plan

No tests, docs-only change.
2024-11-29 21:58:20 -05:00
Charlie Marsh 950855877a
Use constraints in trace rather than irrelevant requires-python (#9529) 2024-11-29 19:25:43 +00:00
Charlie Marsh 58cf93a219
Un-gate `--emit-marker-expression` tests via `--python-platform` (#9528)
These are tedious to update on non-Linux, but I think we can just use
`--python-platform` to make them work on all platforms.
2024-11-29 19:23:12 +00:00
trag1c 9864d23f48
docs: correct wording for multiple sources section (#9504)
## Summary

Corrects calling tags "commits".

## Test Plan

N/A
2024-11-29 12:33:08 -05:00
konsti 772251027e
Make pip install snapshot independent of settings fields (#9522)
When changing something about the settings,
`invalid_pyproject_toml_option_unknown_field` would fail unexpectedly
because the exact list of possible options had changed. Since we're
already testing this list in the settings-related test
`resolve_config_file`, i'm stubbing the exact output here.
2024-11-29 12:32:47 -05:00
Charlie Marsh 57900f33bd
Override `manylinux_compatible` with `--python-platform` (#9526)
## Summary

Closes https://github.com/astral-sh/uv/issues/9521.
2024-11-29 17:26:24 +00:00
Jp b9740d4e16
Align tempfile workspace dependencies with root project (#9524)
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary
While working on potential bug fixes with temporary files on Windows (I
think I am currently ecountering the same issue as #2810)
I noticed that sub-workspaces were not all having the same `tempfile`
version. And they were not relying on the cargo root project dependency.
I don't know at all if it was done on purpose or not.
(I also wanted to override the root dependency with a local source but
it was not possible due to sub-workspaces not relying on the same).

The root lockfile already pinned to the `3.14.0`. Some sub-workspaces
were depending on the `3.12.0`, some others on the `3.14.0`. So I
updated the root `Cargo.toml` to the `3.14.0`.

Feel free to decline if it was done on purpose! No worries at all
🙂

Thanks!

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
All units tests are still passing on my side. Let's see with the
pull-request CI 😄
2024-11-29 12:05:10 -05:00
Charlie Marsh cf20673197
Upgrade to Rust 1.83 (#9511)
## Summary

A lot of good new lints, and most importantly, error stabilizations. I
tried to find a few usages of the new stabilizations, but I'm sure there
are more.

IIUC, this _does_ require bumping our MSRV.
2024-11-29 12:04:22 -05:00
konsti b9b37a9bab
Build backend: Warn when visiting over 10k files (#9523)
Also log how many files we visited in total as a debugging aid.

No test for this one since we don't want to create 10k on disk in our
test suite.
2024-11-29 16:46:11 +01:00
Charlie Marsh 590294a0ac
Propagate markers for recursive extras in resolver (#9509)
## Summary

Closes
https://github.com/astral-sh/uv/issues/9499#issuecomment-2506491209.
2024-11-28 12:40:00 -05:00
Zanie Blue 4a074141b6
Fixup version in prior uninstall instructions (#9485) 2024-11-28 08:00:09 -06:00
konsti b80bd87b99
Don't emit unpinned warning for proxy packages (#9497)
Whenever we see an unpinned proxy package, we know that its base package
is also unpinned and emitted a warning.

Part 1 of 2 for #8155
2024-11-28 12:39:38 +00:00
Charlie Marsh 8a27d4d340
Allow file: URLs to include package names (#9493)
## Summary

Closes https://github.com/astral-sh/uv/issues/9489.
2024-11-27 22:27:07 -05:00
Zanie Blue be651ed32d
Fix typo in entry point docs (#9491) 2024-11-28 02:48:52 +00:00
Charlie Marsh 201dfef780
Insert backslash when appending to system drive (#9488)
## Summary

When you pass a system drive to `Path::join`, Rust doesn't insert a
backslash between the drive and the path itself, so our lookups for
system configuration were failing.

Closes https://github.com/astral-sh/uv/issues/9416.
2024-11-27 19:38:29 -05:00
thirtysix aa688226aa
Fix `--refresh-package` flag mentioned as `--refresh-dependency` (#9486)
## Summary

This PR fixes name of `--refresh-package` flag in cache docs. It's
misspelled as `--refresh-dependency`.

## Test Plan

Deployed docs locally.
2024-11-27 17:14:01 -06:00
Charlie Marsh f1ccbcb065
Consistently enforce requested-vs.-built metadata when retrieving wheels (#9484)
## Summary

We were missing a bunch of edge cases, e.g., the wheel exists in the
cache already.

Closes https://github.com/astral-sh/uv/issues/9480.
2024-11-27 16:51:01 -05:00
Charlie Marsh 4f2b30ca02
Improve error messages for mismatches in `tool.uv.sources` (#9482)
## Summary

Closes https://github.com/astral-sh/uv/issues/9479.
2024-11-27 20:10:39 +00:00
konsti 0b0d0f44f8
Publish: Warn when keyring has no password (#8827)
When trying to upload without a password but with the keyring, check
that the keyring has a password for the upload URL and username and warn
if it doesn't.

Fixes #8781
2024-11-27 20:54:49 +01:00
Charlie Marsh 68e3228f2b
Remove `lxml` from lock test (#9481)
## Summary

Closes https://github.com/astral-sh/uv/issues/9467.
2024-11-27 18:39:38 +00:00
Zanie Blue 95cd8b8b3f
Bump version to 0.5.5 (#9478) 2024-11-27 11:37:39 -06:00
Zanie Blue 7df3ae2585
Add Python implementation example to limited resolution environments docs (#9475)
Closes https://github.com/astral-sh/uv/issues/9473
2024-11-27 09:53:39 -06:00
Zanie Blue 6afb34091c
Expand entry points documentation (#9329) 2024-11-27 09:53:07 -06:00
Andrew Gallant cc6bfa14d1 uv/tests: move `conflicts` tests to its own file
There are already a fair number and I'm planning to add more. And
`lock.rs` is already quite big.

There aren't any new tests or other changes here. This is just moving
tests and trimming down the function names to avoid redundancy in the
names.
2024-11-27 10:51:11 -05:00
Charlie Marsh ee84620e90
Use consistent formatting for build system errors (#9340)
## Summary

These look pretty different from the help / hint messages we show on
resolver failure. I've added color, backticks, and a "hint:" prefix.

Before:

![Screenshot 2024-11-21 at 5 45
40 PM](https://github.com/user-attachments/assets/71ee2551-8fc6-4715-a9d7-68055cd34547)

After:

![Screenshot 2024-11-21 at 6 05
31 PM](https://github.com/user-attachments/assets/24e7fb46-49aa-4b6f-a442-115a71a3414b)
2024-11-27 14:22:53 +00:00
Charlie Marsh 0cd9c542ad
Show an interpreter-focused message for `--target` and `--prefix` (#9373)
## Summary

With `uv pip install --target` and `--prefix`, we (1) should allow
managed Pythons, and (2) should show a different message that's focused
on the interpreter we selected, rather than the environment.
2024-11-27 14:21:46 +00:00
Charlie Marsh 7169b2c427
Respect sources in overrides and constraints (#9455)
## Summary

We still only respect overrides and constraints in the workspace root --
which we may want to change -- but overrides and constraints are now
correctly lowered.

Closes https://github.com/astral-sh/uv/issues/8148.
2024-11-27 13:56:14 +00:00