## Summary
I spent time trying to figure out how to support this but came up empty.
It _seems_ like maybe the `DefaultFields` implementation in
`tracing-subscriber` uses debug formatting for fields...? So if you have
a string with ANSI codes, they end up printing as unformatted values? I
even reverted all our custom formatting in `logging.rs` and saw the same
thing.
Closes https://github.com/astral-sh/uv/issues/15840.
## Summary
Fixes issue #15190 where `uv sync --no-sources` fails to switch from
editable to registry package installations. The problem occurred because
the installer's satisfaction check didn't consider the `--no-sources`
flag when determining if an existing editable installation was
compatible with a registry requirement.
## Solution
Modified `RequirementSatisfaction::check()` to reject non-registry
installations when `SourceStrategy::Disabled` and the requirement is
from registry. Added `SourceStrategy` parameter threading through the
entire call chain from commands to the satisfaction check to ensure
consistent behavior between `uv sync --no-sources` and `uv pip install
--no-sources`.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
We've received several requests to validate that installed wheels match
the current Python platform. This isn't _super_ common, since it
requires that your platform changes in some meaningful way (e.g., you
switch from x86 to ARM), though in practice, it sounds like it _can_
happen in HPC environments. This seems like a good thing to do
regardless, so we now validate that the tags (as recoded in `WHEEL`) are
consistent with the current platform during installs.
Closes https://github.com/astral-sh/uv/issues/15035.
## Summary
This is causing some cyclic dependencies issues for me, because these
can be used in virtually _any_ crate (like `uv-install-wheel`), which
then means that all of `uv-configuration` becomes a dependency, etc. I
think this should be a leaf crate so that we can safely depend on it
anywhere.
## Summary
I've written a reasonably-long comment to explain what's going on here.
We should fix this, but it's better to continue using a
potentially-stale distribution than to panic.
Closes https://github.com/astral-sh/uv/issues/15386.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This PR productionizes an idea I saw in
https://github.com/astral-sh/uv/issues/15248, which was added in Pixi:
https://github.com/prefix-dev/pixi/pull/4247. The core of the idea is
that if we install all build isolation-enabled packages first, and the
build isolation-disabled packages in a second phase, the sync is more
likely to "just work", because if all the build dependencies of the
build isolation-disabled packages are included as dependencies (as is
the case for `flash-attn`, at least), they'll be present.
This isn't really a silver bullet, because it requires that all the
build dependencies are included as first-party dependencies, and if you
have packages that want build isolation to be disabled but rely on other
packages that also require build isolation disabled, that won't work
either. I think `extra-build-dependencies` will be more robust and have
much better caching behavior, but this will get more cases right than
our current behavior, and I don't see any downsides.
Closes https://github.com/astral-sh/uv/issues/15301.
## Summary
With this PR, we track the settings that were used to build a wheel
(`--config-settings`, plus any `extra-build-dependencies` or
`extra-build-variables`) and write those to the `.dist-info` directory
upon install. This then allows us to "reject" already-installed wheels,
if the user changes the build dependencies or `--config-settings` (or,
crucially, if they use `match-runtime = true` and the resolution
changes).
Closes https://github.com/astral-sh/uv/issues/15218.
## Summary
This breaks up a cycle I'm running into in incorporating the build
configuration into our cache keys. This is actually a type that ends up
in the frontend build system, etc., so I think it makes more sense here
anyway (as opposed to `uv-configuration` which tend to be our own
user-facing types).
## Summary
Split the cleanup fixes from https://github.com/astral-sh/uv/pull/15196
into a separate PR for easier review.
This cleans up some minor env var usage / references throughout tests
and runtime code.
## Test Plan
Existing Tests. No functional changes.
## Summary
I should've noticed this during review -- my bad -- but it looks like
after lowering, we're converting back to `uv_pep508::Requirement`. This
is mostly okay, but it's lossy for some lowerings. For example, we lose
index pinning. With this PR, we now preserve the lowered types
(`Requirement`).
Closes https://github.com/astral-sh/uv/issues/15037.
## Summary
We weren't including these in the cache key when constructing the
install plan. We likely still read them from the cache later, but we may
have reported the wrong number of prepares, etc.
If a user specifies `-e /path/to/dir` and `/path/to/dir` in a `uv pip
install` command, we want the editable to "win" (rather than erroring
due to conflicting URLs). Unfortunately, this behavior meant that when
you requested a package as editable and non-editable in conflicting
groups, the editable version was _always_ used. This PR modifies the
requisite types to use `Option<bool>` rather than `bool` for the
`editable` field, so we can determine whether a requirement was
explicitly requested as editable, explicitly requested as non-editable,
or not specified (as in the case of `/path/to/dir` in a
`requirements.txt` file). In the latter case, we allow editables to
override the "unspecified" requirement.
If a project includes a path dependency twice, once with `editable =
true` and once without any `editable` annotation, those are now
considered conflicting URLs, and lead to an error, so I've marked this
change as breaking.
Closes https://github.com/astral-sh/uv/issues/14139.
## Summary
When refactoring the addition PR I accidentally introduced a bug where
the debug message would not be output if the default value is used.
cc @zanieb
## Summary
When installing packages on _very_ slow/overloaded systems it'spossible
to trigger bytecode compilation timeouts, which tends to happen in
environments such as Qemu (especially without KVM/virtio), but also on
systems that are simply overloaded. I've seen this in my Nix builds if I
for example am compiling a Linux kernel at the same time as a few other
concurrent builds.
By making the bytecode compilation timeout adjustable you can work
around such issues. I plan to set `UV_COMPILE_BYTECODE_TIMEOUT=0` in the
[pyproject.nix
builders](https://pyproject-nix.github.io/pyproject.nix/build.html) to
make them more reliable.
- Related issues
* https://github.com/astral-sh/uv/issues/6105
## Test Plan
Only manual testing was applied in this instance. There is no existing
automated tests for bytecode compilation timeout afaict.
Prior to this PR, there were numerous places where uv would leak
credentials in logs. We had a way to mask credentials by calling methods
or a recently-added `redact_url` function, but this was not secure by
default. There were a number of other types (like `GitUrl`) that would
leak credentials on display.
This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials
when logging by default. It takes a maximalist approach, replacing the
use of `Url` almost everywhere. This includes when first parsing config
files, when storing URLs in types like `GitUrl`, and also when storing
URLs in types that in practice will never contain credentials (like
`DirectorySourceUrl`). The idea is to make it easy for developers to do
the right thing and for the compiler to support this (and to minimize
ever having to manually convert back and forth). Displaying credentials
now requires an active step. Note that despite this maximalist approach,
the use of the newtype should be zero cost.
One conspicuous place this PR does not use `DisplaySafeUrl` is in the
`uv-auth` crate. That would require new clones since there are calls to
`request.url()` that return a `&Url`. One option would have been to make
`DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime
annotations all over the codebase. I've created a separate PR based on
this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with
one new clone. We can discuss the tradeoffs there.
Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is
`uv_redacted/lib.rs`, where the newtype is implemented. To make it
easier to review the rest, here are some points of note:
* `DisplaySafeUrl` has a `Display` implementation that masks
credentials. Currently, it will still display the username when there is
both a username and password. If we think is the wrong choice, it can
now be changed in one place.
* `DisplaySafeUrl` has a `remove_credentials()` method and also a
`.to_string_with_credentials()` method. This allows us to use it in a
variety of scenarios.
* `IndexUrl::redacted()` was renamed to
`IndexUrl::removed_credentials()` to make it clearer that we are not
masking.
* We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest`
methods like `.get()` and `.head()`.
* We convert from a `DisplaySafeUrl` to a `Url` when creating a
`uv_auth::Index`. That is because, as mentioned above, I will be
updating the `uv_auth` crate to use this newtype in a separate PR.
* A number of tests (e.g., in `pip_install.rs`) that formerly used
filters to mask tokens in the test output no longer need those filters
since tokens in URLs are now masked automatically.
* The one place we are still knowingly writing credentials to
`pyproject.toml` is when a URL with credentials is passed to `uv add`
with `--raw`. Since displaying credentials is no longer automatic, I
have added a `to_string_with_credentials()` method to the `Pep508Url`
trait. This is used when `--raw` is passed. Adding it to that trait is a
bit weird, but it's the simplest way to achieve the goal. I'm open to
suggestions on how to improve this, but note that because of the way
we're using generic bounds, it's not as simple as just creating a
separate trait for that method.
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
## Summary
I don't know if I actually want to commit this, but I did it on the
plane last time and just polished it off (got it to compile) while
waiting to board.
## Summary
This crate is for standards-compliant types, but this is explicitly a
type that's custom to uv. It's also strange because we kind of want to
reference `IndexUrl` on the registry type, but that's in a crate that
_depends_ on `uv-pypi-types`, which to me is a sign that this is off.
## Summary
I think this is reasonable to change. Right now, if you're on Python
3.11, the resolver returns `multiprocess-0.70.17-py311-none-any.whl`,
but `multiprocess-0.70.17-py310-none-any.whl` is in the cache, we'll
reuse `multiprocess-0.70.17-py310-none-any.whl` (since it _is_
compatible with Python 3.11).
Instead, we now _require_ the cached wheel to match the wheel returned
by the resolver.
Closes https://github.com/astral-sh/uv/issues/12273.
## Summary
This ended up being more involved than expected. The gist is that we
setup all the packages we want to reinstall upfront (they're passed in
on the command-line); but at that point, we don't have names for all the
packages that the user has specified. (Consider, e.g., `uv pip install
.` -- we don't have a name for `.`, so we can't add it to the list of
`Reinstall` packages.)
Now, `Reinstall` also accepts paths, so we can augment `Reinstall` based
on the user-provided paths.
Closes#12038.
## Summary
This PR adds support for `SitePackages::satisfies` with unnamed
overrides and requirements.
The main challenge here was cases like: you have a `requirements.in`
with `git+https://github.com/pallets/flask` in it, and an
`overrides.txt` with `flask==2.0.0` in it. You _need_ to include
`flask==2.0.0`, but you can't know that without resolving the unnamed
URL requirement (since overrides only take effect when the package is
included, like constraints).
We now make the assumption that any unnamed overrides _are_ relevant,
for the purpose of the satisfies check. This is conservative, but this
whole check is an optimization anyway.
## Summary
Small omission I noticed last night. This was overly strict (so, didn't
lead to any incorrect behavior; more that we did unnecessary work in
some cases).
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
Closes#2410
<!-- What's the purpose of the change? What does it do, and why? -->
This changes the name of files in `wheels` bucket to use a hash instead
of the wheel name as to not exceed maximum file length limit on various
systems.
This only addresses the primary concern of #2410. It still does _not_
address:
- Path limit of 260 on windows:
https://github.com/astral-sh/uv/issues/2410#issuecomment-2062020882
To solve this we need to opt-in to longer path limits on windows
([ref](https://github.com/astral-sh/uv/issues/2410#issuecomment-2150532658)),
but I think that is a separate issue and should be a separate MR.
- Exceeding filename limit while building a wheel from source
distribution
As per my understanding, this is out of uv's control. Name of the output
wheel will be decided by build-backend used by the project. For wheels
built from source distribution, pip also uses the wheel names in cache.
So I have not touched `sdists` cache.
I have added a `filename: WheelFileName` field in `Archive`, so we can
use it while indexing instead of relying on the filename on disk.
Another way to do this was to read `.dist-info/WHEEL` and
`.dist-info/METADATA` and build `WheelFileName` but that seems less
robust and will be slower.
## Test Plan
<!-- How was it tested? -->
Tested by installing `yt-dlp`, `httpie` and `sqlalchemy` and verifying
that cache files in `wheels` bucket use hash.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Solving spent a chunk of its time just converting resolutions, the left
two blocks:
These blocks are `ResolverOutput::from_state` with 1.3% and
`ForkState::into_resolution` with 4.1% of resolver thread runtime for
apache airflow universal.
We reduce the overhead spent in those functions, to now 1.1% and 2.1% of
resolver time spend in those functions by:
Commit 1: Replace the hash set for the edges with a vec in
`ForkState::into_resolution`. We deduplicate edges anyway when
collecting them, and the hash-and-insert was slow.
Commit 2: Reduce the distribution clonign in
`ResolverOutput::from_state` by using an `Arc`.
The same profile excerpt for the resolver with the branch (note that
there is now an unrelated block between the two we optimized):
Wall times are noisy, but the profiles show those changes as
improvements.
```
$ hyperfine --warmup 2 "./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal" "./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal"
Benchmark 1: ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 99.1 ms ± 3.8 ms [User: 111.8 ms, System: 115.5 ms]
Range (min … max): 93.6 ms … 110.4 ms 29 runs
Benchmark 2: ./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal
Time (mean ± σ): 97.1 ms ± 4.3 ms [User: 114.8 ms, System: 112.0 ms]
Range (min … max): 90.9 ms … 112.4 ms 29 runs
Summary
./uv-branch pip compile --no-progress scripts/requirements/airflow.in --universal ran
1.02 ± 0.06 times faster than ./uv-main pip compile --no-progress scripts/requirements/airflow.in --universal
```
Instead of always using all available threads for bytecode compilation,
respect `UV_CONCURRENT_INSTALLS`, so the parallelism is configurable
instead of hardcoded. We reuse the install limit since bytecode
compilation only runs after install.
Initially, we were limiting Git schemes to HTTPS and SSH as only
supported schemes. We lost this validation in #3429. This incidentally
allowed file schemes, which apparently work with Git out of the box.
A caveat for this is that in tool.uv.sources, we parse the git field
always as URL. This caused a problem with #11425: repo = { git =
'c:\path\to\repo', rev = "xxxxx" } was parsed as a URL where c: is the
scheme, causing a bad error message down the line.
This PR:
* Puts Git URL validation back in place. It bans everything but HTTPS,
SSH, and file URLs. This could be a breaking change, if users were using
a git transport protocol were not aware of, even though never
intentionally supported.
* Allows file: URL in Git: This seems to be supported by Git and we were
supporting it albeit unintentionally, so it's reasonable to continue to
support it.
* It does not allow relative paths in the git field in tool.uv.sources.
Absolute file URLs are supported, whether we want relative file URLs for
Git too should be discussed separately.
Closes#3429: We reject the input with a proper error message, while
hinting the user towards file:. If there's still desire for relative
path support, we can keep it open.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
## Summary
We currently enforce that if you do `uv pip install
./dist/iniconfig-1.0.0.tar.gz`, the build _must_ produce a wheel like
`iniconfig-1.0.0-py3-none-any.whl` (i.e., the name and version must
match). It turns out some packages produce a wheel that has a local
suffix on it, like `vllm`. This PR makes the check a little more
permissive in that we now accept `1.0.0` or that version with a local
suffix (e.g., `1.0.0+cpu`). I don't love this practice, but we already
relaxed this check when _installing_ a wheel, so this seems reasonable:
5e15881dcc/crates/uv-install-wheel/src/install.rs (L50-L52)
Note that this is _still_ stricter than pip. pip seems to only require
that the package name is the same (i.e., `iniconfig` matches
`iniconfig`; but they'll happily install a wheel like
`iniconfig-2.0.0-py3-none-any.whl` given
`./dist/iniconfig-1.0.0.tar.gz`).
Closes https://github.com/astral-sh/uv/issues/11038.
samypr100
Charlie Marsh
chisato
Zanie Blue
adisbladis
John Mumm
konsti
Aria Desires
Ankit Saini