Commit Graph

3136 Commits

Author SHA1 Message Date
renovate[bot]
3b488d8635 Update Rust crate cargo-util to v0.2.13 (#5536) 2024-07-28 20:30:28 -04:00
renovate[bot]
f8f82b354d Update Rust crate assert_fs to v1.1.2 (#5535) 2024-07-28 20:30:21 -04:00
renovate[bot]
9164d5cf97 Update Rust crate anstream to v0.6.15 (#5533) 2024-07-28 20:30:14 -04:00
Pavel Dikov
cb47aed9de feat(venv): add relocatable flag (#5515)
## Summary

Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:

* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
  `.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.

The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).

Fixes: #3863

## Test Plan

* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
2024-07-29 00:10:11 +00:00
Charlie Marsh
3626d08cca Enable benchmarking of uv tool and pipx (#5531)
## Summary

Closes https://github.com/astral-sh/uv/issues/5263.
2024-07-28 23:27:14 +00:00
Charlie Marsh
44a77a04d0 Move bench directory to benchmark (#5529)
## Summary

Removes the legacy `benchmark` directory (we'll always have it in Git)
and renames `bench` to `benchmark` for clarity. Fixes a variety of
commands and references.
2024-07-28 22:03:52 +00:00
Charlie Marsh
d7c79182ea Extend Ruff configuration to sort imports (#5528) 2024-07-28 21:49:28 +00:00
Charlie Marsh
9732922929 Migrate benchmark setup to uv (#5526) 2024-07-28 21:37:57 +00:00
Charlie Marsh
2d5c166642 Compare simplified paths in Windows exclusion tests (#5525)
## Summary

Closes https://github.com/astral-sh/uv/issues/5521.
2024-07-28 17:26:46 -04:00
Charlie Marsh
efbc9fb78d Add support for benchmarking uv sync and uv lock (#5524)
## Summary

This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.

Part of: https://github.com/astral-sh/uv/issues/5263.

## Test Plan

For example:

```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
2024-07-28 21:09:08 +00:00
Mark Ture
12e92b7718 Fix typo in projects.md (#5523) 2024-07-28 21:05:26 +01:00
Charlie Marsh
83412837e5 Warn, but don't error, when encountering tilde .dist-info directories (#5520)
## Summary

Closes https://github.com/astral-sh/uv/issues/3668.
2024-07-28 19:13:06 +00:00
Charlie Marsh
b0c841ee3b Ban --no-cache with --link-mode=symlink (#5519)
## Summary

Closes https://github.com/astral-sh/uv/issues/5360.
2024-07-28 15:01:17 -04:00
Krishnan Chandra
4b4128446d Support xz compressed packages (#5513)
## Summary

Closes #2187.

The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.

It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.

## Test Plan

Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`

(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-28 18:37:48 +00:00
Charlie Marsh
caf01735fa Avoid warning users for missing self-extra lower bounds (#5518)
## Summary

Closes https://github.com/astral-sh/uv/issues/5227.
2024-07-28 18:35:18 +00:00
Charlie Marsh
88340fbd0d Remove some unused methods (#5512) 2024-07-28 17:20:12 +00:00
Charlie Marsh
1734c7ed50 Use existing METADATA parser in wheel installer (#5508) 2024-07-27 14:38:16 -04:00
Jo
ae11317cc0 Make pip list --editable conflicts with --exlcude-editable (#5506)
## Summary

I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.

## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'

Usage: uv.exe pip list --editable

For more information, try '--help'.

```
2024-07-27 08:26:46 -04:00
Charlie Marsh
866d844977 Allow uv pip install for unmanaged projects (#5504)
## Summary

Just an oversight.

Closes: https://github.com/astral-sh/uv/issues/5500.
2024-07-27 02:10:18 +00:00
Charlie Marsh
24859bd3ee Upgrade to Rust 1.80.0 (#5472) 2024-07-27 01:49:47 +00:00
Charlie Marsh
3ea5e16e96 Respect reinstalls in cached environments (#5499)
## Summary

Closes #5493.
2024-07-27 01:36:58 +00:00
Charlie Marsh
8f16f1b746 Remove serialize traits from verbatim URL (#5501) 2024-07-27 00:51:24 +00:00
Charlie Marsh
4f3dde34dc Use 666 rather than 644 for default permissions (#5498)
## Summary

I don't know why I used 644 here. 666 is the actual default:
7c2012d0ec/library/std/src/sys/pal/unix/fs.rs (L1069)

Closes https://github.com/astral-sh/uv/issues/5496.
2024-07-27 00:08:52 +00:00
Charlie Marsh
623ba3885f Add some missing reinstall-refresh calls (#5497)
## Summary

Turns out I missed a few of these.
2024-07-27 00:01:40 +00:00
Charlie Marsh
561625ed8c Use hasher to compute resolution hash (#5495)
## Summary

Addressing one TODO. This should be more efficient.
2024-07-26 23:24:09 +00:00
Ahmed Ilyas
e8d7c0cb58 Editable installs for uv tool (#5454)
## Summary

Resolves #5436. 

## Test Plan

`cargo test` 

```console
❯ ./target/debug/uv tool install -e ~/black
warning: `uv tool install` is experimental and may change without warning
Resolved 6 packages in 894ms
   Built black @ file:///Users/ahmedilyas/black
Prepared 1 package in 468ms
Installed 6 packages in 6ms
 + black==24.4.3.dev23+g7e2afc9 (from file:///Users/ahmedilyas/black)
 + click==8.1.7
 + mypy-extensions==1.0.0
 + packaging==24.1
 + pathspec==0.12.1
 + platformdirs==4.2.2
Installed 2 executables: black, blackd
```

venv has the `.pth` files.
```console
❯ eza /Users/ahmedilyas/Library/Application\ Support/uv/tools/black/lib/python3.12/site-packages/
_black.pth       _virtualenv.py                         click                  mypy_extensions-1.0.0.dist-info  packaging                 pathspec                   platformdirs
_virtualenv.pth  black-24.4.3.dev23+g7e2afc9.dist-info  click-8.1.7.dist-info  mypy_extensions.py               packaging-24.1.dist-info  pathspec-0.12.1.dist-info  platformdirs-4.2.2.dist-info
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-26 16:30:15 -04:00
Andrew Gallant
2186e967f6 uv-resolver: fix basic case of overlapping markers
Consider the following packse scenario:

```toml
[root]
requires = [
  "a>=1.0.0 ; python_version < '3.10'",
  "a>=1.1.0 ; python_version >= '3.10'",
  "a>=1.2.0 ; python_version >= '3.11'",
]

[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```

On current `main`, this produces a dependency on `a` that looks like
this:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```

But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a" },
]
```

That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.

The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.

While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.

For the packse scenario test: https://github.com/astral-sh/packse/pull/206
2024-07-26 12:06:37 -07:00
Charlie Marsh
6901a14aa0 Bump version to v0.2.30 (#5486) 0.2.30 2024-07-26 13:36:05 -04:00
konsti
f1eda3590b Update to packse 0.3.31 (#5441)
Update packse to 0.3.31, adding the instability scenarios.
2024-07-26 15:39:29 +00:00
konsti
cb505d24f8 Unify resolutions only during graph building (#5479)
With our previous eager union, we were losing the fork markers. We now
carry this information into the resolution graph construction and, in
the next step, can read the markers there.

Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198
2024-07-26 16:29:48 +02:00
Andrew Gallant
77b005244d uv-resolver: propagate markers to sibling dependencies in forks
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.

This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.

In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.

So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.

This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.

This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.

Fixes #5086

Partially addresses #4732
2024-07-26 07:28:20 -07:00
Andrew Gallant
412780fd99 uv/tests: add regression test from #5086
The snapshot saved here is wrong, but we'll update it in a subsequent
commit.
2024-07-26 07:28:20 -07:00
Andrew Gallant
cd1fc7c9a3 uv-normalize: make "name" types implement Default
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
2024-07-26 07:28:20 -07:00
Andrew Gallant
7fce59e4bc pep508: some simplification in 'and' and 'or'
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
2024-07-26 07:28:20 -07:00
Charlie Marsh
8b8f34ac21 Avoid canonicalizing executables on Windows (#5446)
## Summary

If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.

This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.

Closes: https://github.com/astral-sh/uv/issues/5440.
2024-07-26 08:57:33 -04:00
Charlie Marsh
967fcdbb83 Update bundled Pythons to include stripped variants (#5469)
## Summary

The previous update used a slightly botched release.
2024-07-26 00:34:45 +00:00
Charlie Marsh
d51b429837 Add --no-config to replace --isolated (#5463)
## Summary

I'll deprecate `--isolated` separately, since it _is_ still used for
some other behaviors.

Closes #5428.
2024-07-25 19:58:36 -04:00
Charlie Marsh
8cb22f5500 Gate FromStr to pep508 feature (#5464)
## Summary

This is triggering when you run without `--all-features` (but we always
use `--all-features` in CI, so it went unnoticed).
2024-07-25 23:54:07 +00:00
Charlie Marsh
e4d1039f49 Use sitecustomize.py to implement environment layering (#5462)
## Summary

After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.

This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.

Closes https://github.com/astral-sh/uv/issues/5459.
2024-07-25 19:32:44 -04:00
Charlie Marsh
6f45403d31 Infer missing .exe in Windows Python discovery (#5456)
## Summary

Closes https://github.com/astral-sh/uv/issues/5445.
2024-07-25 16:57:38 -04:00
Charlie Marsh
1e965b47c3 Set standard permissions for temporary files (#5457)
## Summary

Closes https://github.com/astral-sh/uv/issues/5435.

## Test Plan

Before:

```
❯ ls -l .venv/lib/python3.12/site-packages/httpx-0.27.0.dist-info
total 48
-rw-------  1 crmarsh  staff     2 Jul 25 14:21 INSTALLER
-rw-r--r--  1 crmarsh  staff  7184 Jul 23 23:20 METADATA
-rw-r--r--  1 crmarsh  staff  2541 Jul 25 14:21 RECORD
-rw-------  1 crmarsh  staff     0 Jul 25 14:21 REQUESTED
-rw-r--r--  1 crmarsh  staff    87 Jul 23 23:20 WHEEL
-rw-r--r--  1 crmarsh  staff    37 Jul 23 23:20 entry_points.txt
drwxr-xr-x  3 crmarsh  staff    96 Jul 25 14:21 licenses
```

After:

```
❯ ls -l .venv/lib/python3.12/site-packages/flask-3.0.3.dist-info/
total 48
-rw-r--r--  1 crmarsh  staff     2 Jul 25 14:21 INSTALLER
-rw-r--r--  1 crmarsh  staff  1475 Jul 25 14:21 LICENSE.txt
-rw-r--r--  1 crmarsh  staff  3177 Jul 25 14:21 METADATA
-rw-r--r--  1 crmarsh  staff  2565 Jul 25 14:21 RECORD
-rw-r--r--  1 crmarsh  staff     0 Jul 25 14:21 REQUESTED
-rw-r--r--  1 crmarsh  staff    81 Jul 25 14:21 WHEEL
-rw-r--r--  1 crmarsh  staff    40 Jul 25 14:21 entry_points.txt
```
2024-07-25 20:50:30 +00:00
Charlie Marsh
75a042d5ff Allow distributions to be absent in deserialization (#5453)
## Summary

Closes https://github.com/astral-sh/uv/issues/5434.
2024-07-25 18:01:41 +00:00
Zanie Blue
6eb8f85668 Update documentation sections (#5452)
Reframes "the low-level interface" as "the pip interface"
Adds indexes to all sections
Renames "commercial indexes" to "alternative indexes"
2024-07-25 12:37:22 -05:00
Charlie Marsh
a089e20a35 Use stripped variants by default in Python install (#5451)
## Summary

Will file a separate ticket to support `--debug`.

Closes https://github.com/astral-sh/uv/issues/5447.
2024-07-25 17:29:31 +00:00
Ibraheem Ahmed
4da34a6e2e Reorganize pep508 marker module (#5433)
## Summary

Just moves some code around into separate modules, the current file is a
bit too large to navigate.
2024-07-25 15:22:46 +00:00
Andrew Gallant
7dff3d7dfe uv/tests: updates snapshots for main
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
2024-07-25 07:42:26 -07:00
InSync
d17b18ee1e Minor consistency fixes for code blocks (#5437)
...as well as some typo fixes. I verified the changes manually.
2024-07-25 09:23:22 -05:00
Zanie Blue
42e76e2545 Prefer "lockfile" to "lock file" (#5427)
Closes https://github.com/astral-sh/uv/issues/5415
2024-07-25 09:22:36 -05:00
Charlie Marsh
d0919329fd Make --reinstall imply --refresh (#5425)
## Summary

It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.

This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.

Closes https://github.com/astral-sh/uv/issues/5424.
2024-07-25 09:45:58 -04:00
Charlie Marsh
4d9098a1d7 Cache metadata for source tree dependencies (#5423)
## Summary

This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).

Now that we support persistent configuration, users should add:

```toml
[tool.uv]
reinstall = ["foo"]
```

If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).

Closes https://github.com/astral-sh/uv/issues/5420.
2024-07-25 09:45:52 -04:00