## Summary
This PR enables `uv sync --all-packages` to sync all packages in a
workspace. It removes a common use-case for the legacy non-`[project]`
packages that we're trying to move away from.
Closes https://github.com/astral-sh/uv/issues/8724.
This PR fixes a bug where it was possible for dependencies to be
included in a final resolution with markers that always evaluate to
false. Specifically, `python_version < '0'`.
While we do filter based on Python markers during forking, it turns out
that the markers for each fork are "combined" *after* this filtering
step. But the process of combination can result in a more specific
marker that is always false for the configured Python requirement. This
could result in dependencies with markers that are always false (like
`python_version < '0'`) appearing in the resolution.
The first commit in this PR adds a regression test (with an undesirable
result), and the second commit fixes the regression and updates the
test.
Fixes#8676
This still utilizes the RFC 2822 datetime formatter, but utilizes new
methods [added in jiff 0.1.14] to emit timestamps in a format strictly
compatible with RFC 9110.
It seems like most HTTP servers were pretty flexible and supported RFC
2822 datetime formats, but #8747 shows at least one case where that
isn't true. Given that the [MDN docs prescribe RFC 9110], we defer to
them.
Fixes#8747
[added in jiff 0.1.14]: https://github.com/BurntSushi/jiff/pull/154
[MDN docs prescribe RFC 9110]:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
## Summary
It turns out that when locking, we were only taking the groups from the
root `pyproject.toml` into account, and ignoring groups that were only
defined in a workspace member.
## Summary
By default, `uv tree` shows the full workspace, not _just_ the root. If
the root depended on a workspace member as a dev dependency, then we'd
still show it as `(group: dev)` in `uv tree` even if you passed
`--no-dev`, because we weren't filtering the edges in the right place.
This is still somewhat confusing, because if `root` depends on workspace
member `child` as a dev dependency, `uv tree --no-dev` still shows both.
Closes https://github.com/astral-sh/uv/issues/8719.
Incorporating #8637 into #8458
- Adds `python-managed` feature selection to Windows CI for `python
install` tests
- Adds trampoline sniffing utilities to `uv-trampoline-builder`
- Uses a trampoline to install Python executables into the `PATH` on
Windows
Pulling out of https://github.com/astral-sh/uv/pull/8650 for
readability.
Trying to clean this up to simplify extensions in the future. This is
not a strict refactor, there are behavioral changes here.
- Adds some structs for managing state.
- Addresses some likely inconsistent behavior for weird edge-cases.
- We fill platform information before checking if a request is
satisfied.
- We error earlier if we can't find a download for the request, i.e.,
even if you somehow have it installed.
- Only reports versions as uninstalled if a download actually replaces
them.
- Moves some of the default output to tracing messages.
- Even if an installation was already satisfied, we'll check that it is
setup properly
When resolving workspace dependencies (from one workspace member to
another) from a workspace that's in git, we need to emit these
transitive dependencies as git dependencies, not path dependencies as
all other workspace deps. This fixes a bug where we would treat them as
path dependencies inside the checkout directory, leading either to
clashes (between a local path and another direct git dependency) or
invalid lockfiles (referencing the checkout dir in the lockfile when we
should be referencing the git repo).
Fixes#8087Fixes#4920Fixes#3936 since we needed that information anyway
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Updates `uv python install` to link `python3.x` in the executable
directory (i.e., `~/.local/bin`) to the the managed interpreter path.
Includes
- #8569
- #8571
Remaining work
- #8663
- #8650
- Add an opt-out setting and flag
- Update documentation
## Summary
These use really heavy test packages, like SciPy, NumPy, scikit-learn --
and packages with large dependency trees, like packse.
I removed a few redundant tests, and replaced the tests with smaller
packages.
Closes https://github.com/astral-sh/uv/issues/8674.
Previously, when doing a `uv pip` resolution, we would only return the
first entry in the map. But there should only ever be one entry, or else
we would have incompatible dependencies. So we can collapse the case
with the "one universal fork" case.
(I found this while doing some refactoring of how we handle forking, and
collapsing these cases simplifies some of that refactoring work.)
Previously, when tests were run in `~/.local/share/uv`, the behavior of
some tests could be impacted by a git repository in `~` (as on my
system). To avoid this, we set `GIT_CEILING_DIRECTORIES` to forcefully
prevent git from climbing out of its test directory to look for parent
git repositories.
When this code was written, we didn't have "proper" disjointness checks,
and so simple equality was used instead. Arguably disjointness checks
are
more correct, and this would also simplify some case analysis in an
ongoing
refactor.
Currently, our trampoline is used to convert `<command> [args]` to
`python <command> [args]` for script entrypoints installed into virtual
environments. For #8458, it'd be nice to convert a shim `python3.12
[args]` to `python [args]`. Here, we modify the trampolines to support
this use-case.
The only change we really need here is to avoid injecting `<command>`
into the child process. We change the "magic number" at the end of the
trampoline executables from `UVUV` to `UVSC` and `UVPY` which define
"script" and "python" variants to the trampoline. We then omit the
`<command>` injection in the latter case. We also omit writing the zip
script payload.
To support construction of the new variant, a new
`uv-trampoline-builder` crate is introduced — this avoids requirements
on `uv-install-wheel` in future work. I also use `uv-trampoline-builder`
to consolidate some of the test setup for `uv-trampoline`.
There should be no backwards compatibility concerns, since trampolines
are fully self-referential.
I rebased to fix the commits at the end, as this took many iterations to
get working via CI. This should roughly be reviewable by commit if you
prefer.
## Summary
Unfortunately, it looks like we lost
https://github.com/astral-sh/uv/pull/8501 somewhere in a bad rebase.
This PR re-adds the change, with compatibility for those lockfiles
created in v0.4.27. I'm not certain we should actually merge this. It
might be less painful and confusing to just bite the bullet on the
change.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Refs:
- #8626
## Summary
Current documentation incorrectly suggests that the macOS cache
directory location is `$HOME/Library/Caches/uv`, but that changed in:
- #5806
Updates docs to say this instead:
> <p>Defaults to <code>$HOME/.cache/uv</code> on macOS,
<code>$XDG_CACHE_HOME/uv</code> or <code>$HOME/.cache/uv</code> on
Linux, and <code>%LOCALAPPDATA%\uv\cache</code> on Windows. The <code>uv
cache dir</code> command will show the location of the cache
directory.</p>
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
The changes in this commit introduce the `UV_NO_PROGRESS` environment
variable as an alternative way to control progress output suppression in
uv-cli, equivalent to using the `--no-progress` flag. This enhancement
simplifies configuration in CI environments and automated scripts by
eliminating the need to detect whether the script is running in a CI
environment.
Previously, disabling progress output required either passing the
`--no-progress` flag directly or implementing script logic to detect CI
environments and conditionally add the flag. With this change, users can
now simply set `UV_NO_PROGRESS=true` in their environment to achieve the
same effect.
The changes include:
- Adding the `UV_NO_PROGRESS` environment variable to the `EnvVars`
struct in `crates/uv-static/src/env_vars.rs`.
- Updating the `GlobalArgs` struct in `crates/uv-cli/src/lib.rs` to
include a new `no_progress` field that is bound to the `UV_NO_PROGRESS`
environment variable.
- Adding documentation for the new `UV_NO_PROGRESS` environment variable
in `docs/configuration/environment.md`.
## Test Plan
After creating a uv project using `uv init` in a temp directory in this
project:
```
cargo run cache clean && cargo run venv && UV_NO_PROGRESS=false cargo run sync
cargo run cache clean && cargo run venv && cargo run sync
```
produce the expected default behavior
```
cargo run cache clean && cargo run venv && UV_NO_PROGRESS=false cargo run sync
```
produces the same behavior as having the `--no-progress` flag.
## Summary
This PR removes an unnecessary `return` statement from Rust code in the
Maturin template. This makes the generated code more idiomatic and
prevents a Clippy warning.
## Test Plan
This change was tested both in unit tests and by manually creating a
Maturin project.
The shared arguments were resetting the `UV_PYTHON_INSTALL_DIR`,
breaking the intent of the test.
Added some uninstall tests too, needed later for #8571
## Summary
It turns out we were omitting empty dependency groups from the lockfile
metadata, which was then causing us to reject locks when empty groups
were defined.
We now include them (that section of the lock is meant to be a true
representation of the metadata, and an empty-but-defined group is
different from an absent group), though we can ignore them for
validation, since it doesn't affect any behavior.
Closes https://github.com/astral-sh/uv/issues/8581.
## Summary
We already support `tool.uv.dev-dependencies` in the legacy
non-`[project]` projects. This adds equivalent support for
`[dependency-groups]`, e.g.:
```toml
[tool.uv.workspace]
[dependency-groups]
lint = ["ruff"]
```
## Summary
`uv add --dev` now updates the `dependency-groups.dev` section, rather
than `tool.uv.dev-dependencies` -- unless the dependency is already
present in `tool.uv.dev-dependencies`.
`uv remove --dev` now removes from both `dependency-groups.dev` and
`tool.uv.dev-dependencies`.
`--dev` and `--group dev` are now treated equivalently in `uv add` and
`uv remove`.
This PR adds support for `tool.uv.default-groups`, which defaults to
`["dev"]` for backwards-compatibility. These represent the groups we
sync by default.
## Summary
We have to iterate over all user-defined dependencies here. We were
missing the new `[dependency-groups]` section.
Part of https://github.com/astral-sh/uv/pull/8272.
Part of #8090
Unblocks https://github.com/astral-sh/uv/pull/8274
Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
Part of #8090
Adds the ability to read group inclusions (`include-group = <name>`) in
the `pyproject.toml`. Resolves groups into concrete dependencies for
resolution.
See https://github.com/astral-sh/uv/pull/8110 for a bit more commentary
on deferred work.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Part of #8090
Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.
There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.
Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
Part of #8090
Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.
Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
## Summary
These two sentences in the docs for `--publish-url` seem to basically be
duplicates:
3eda248ef5/crates/uv-cli/src/lib.rs (L4616-L4618)
I found the first to be easier to read, so this commit removes the
second.
## Test Plan
No tests, change is docs-only.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
For example, in:
```toml
[tool.uv]
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
We can just omit `[tool.uv]`.
## Summary
We were including dependencies that were only included by a dependency
that isn't relevant on the current platform (i.e., we were enforcing the
"current environment" at one level, but not transitively).
Closes https://github.com/astral-sh/uv/issues/8516.
## Summary
Historically, we haven't enforced schema versions. This PR adds a
versioning policy such that, if a uv version writes schema v2, then...
- It will always reject lockfiles with schema v3 or later.
- It _may_ reject lockfiles with schema v1, but can also choose to read
them, if possible.
(For example, the change we proposed to rename `dev-dependencies` to
`dependency-groups` would've been backwards-compatible: newer versions
of uv could still read lockfiles that used the `dev-dependencies` field
name, but older versions should reject lockfiles that use the
`dependency-groups` field name.)
Closes https://github.com/astral-sh/uv/issues/8465.
With a change like https://github.com/astral-sh/uv/pull/8458, we really
need tests for these.
I'm just going to take the possible performance hit of these slow tests
and deal with optimizing them separately.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Fix the flag description: `to detect and with` --> `to detect packages
with`
This PR adds support for `uv lock --dry-run`, as described in issue
#6408.
One thing to note: this functionality, as implemented, isn't limited to
`-U` (if someone adds a dependency to the project's `pyproject.toml`,
the plan will include these changes).
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Previously, `uv tree --package` had some strange behavior due to how we
were computing the root nodes. This PR refactors the entire
implementation to use `petgraph` so we can do proper operations on a
graph structure.
Closes https://github.com/astral-sh/uv/issues/8382.
## Summary
This PR delays the removal of an existing version after downloading the
new version when running `uv python install --reinstall`.
If the download fails, we can keep the existing version working.
## Test Plan
```console
$ cargo run -- python install 3.13
$ cargo run -- python install --reinstall 3.13 # when downloading, `ctrl-c` to interrupt
$ cargo run -- python list
```
## Summary
Instead of creating a new entry, we should reuse the existing entry (to
preserve decor); similarly, we should avoid overwriting fields that are
already "correct".
Closes https://github.com/astral-sh/uv/issues/8483.
## Summary
We no longer need this struct; we bumped the cache bucket version
anyway, so the `Timestamp` variant is never encountered. This means we
get real Serde error messages.
Closes https://github.com/astral-sh/uv/issues/8488.
## Summary
This is part of making
https://github.com/astral-sh/uv/issues/7299#issuecomment-2385286341
better. You can now use `tool.uv.dependency-metadata` for direct URL
requirements. Unfortunately, you _must_ include a version, since we need
one to perform resolution.
## Summary
Look for a system level uv.toml config file under `/etc/uv/uv.toml` or
`C:\ProgramData`.
This PR is to address #6742 and start a conversation.
## Test Plan
This was tested locally manually on MacOS. I am happy to contribute
tests once we settle on the approach.
cc @thatch
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
If the user has an upper-bound in a `requires-python`, we don't
correctly narrow it during resolution. We should be narrowing based on
the intersection.
Closes#8297.
## Summary
The desired behavior for `uv tree` and `uv pip list` with `-q | --quiet`
flag is
https://github.com/astral-sh/uv/issues/8379#issuecomment-2425093709 to
still produce output. This is implemented here.
Closes https://github.com/astral-sh/uv/issues/8379
## Test Plan
Use `uv tree -q` or `uv pip list -q` on any uv project setup and expect
the corresponding output.
Added tests for that as well.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Now, we use four space (rather than one space) for cases like:
```toml
dependencies = [ # comment 0
# comment 1
"anyio==3.7.0", # comment 2
# comment 3
]
```
## Summary
Rather than relying on the distribution and package URL being the same
(which isn't true for Git dependencies), we can just use the
intersection of the markers directly.
Closes https://github.com/astral-sh/uv/issues/8381.
Cherry-picked from #8347
Might fix https://github.com/astral-sh/uv/issues/6940 — I'm not seeing a
failure over there after this change. I think there may be some problem
with concurrent reads of junctioned files on the DevDrive? It's really
hard to say.
We might lose some important test coverage with this change. I'm not
sure what to do about that either.
Cherry-picked from https://github.com/astral-sh/uv/pull/8347
Seems generally really helpful to see the unfiltered snapshot when a
test fails. Especially when debugging filters on Windows.
## Summary
This PR is is to address the problem when the same-line comments in
`pyproject.toml` could be found in unpredictable positions after `uv
add` or `remove` reformats the `pyproject.toml` file.
Introduced the `Comment` structure in `pyproject_mut` module to
distinguish "same-line" comments and "full-line" comments while
reformatting, because logic for them differs.
Sorry, the implementation could be clumsy, I'm just learning Rust, but
it seems to work 😅
Closes https://github.com/astral-sh/uv/issues/8343
## Test Plan
Added the new test:
`add_preserves_comments_indentation_and_sameline_comments`
To test followed the actions from the issue ticket
https://github.com/astral-sh/uv/issues/8343
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Going forward, we're going to provide better versioning guarantees
around using the same cache across multiple uv versions, so this PR
updates the docs to reflect that. It also bumps the `sdists-` version to
fix the inconvenience demonstrated in
https://github.com/astral-sh/uv/issues/8367.
Closes https://github.com/astral-sh/uv/issues/8367.
## Summary
Resolves#7685
## Test Plan
```console
$ echo "this is an invalid netrc" > .netrc
$ NETRC=.netrc cargo run -- pip install anyio --index-url https://pypi-proxy.fly.dev/basic-auth/simple --strict -v
DEBUG uv 0.4.24 (f4d5fba61 2024-10-19)
DEBUG Searching for default Python interpreter in system path or `py` launcher
DEBUG Found `cpython-3.11.2-windows-x86_64-none` at `D:\Projects\Rust\uv\.venv\Scripts\python.exe` (virtual environment)
DEBUG Using Python 3.11.2 environment at .venv
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: anyio
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.2
DEBUG Solving with target Python version: >=3.11.2
DEBUG Adding direct dependency: anyio*
DEBUG No cache entry for: https://pypi-proxy.fly.dev/basic-auth/simple/anyio/
WARN Error reading netrc file: parsing error: bad toplevel token 'this' (line 1) in the file '.netrc'
DEBUG Searching for a compatible version of anyio (*)
DEBUG No compatible version found for: anyio
× No solution found when resolving dependencies:
╰─▶ Because anyio was not found in the package registry and you require anyio, we can conclude that your
requirements are unsatisfiable.
hint: An index URL (https://pypi-proxy.fly.dev/basic-auth/simple) could not be queried due to a lack of valid
authentication credentials (401 Unauthorized).
DEBUG Released lock at `D:\Projects\Rust\uv\.venv\.lock`
error: process didn't exit successfully: `target\debug\uv.exe pip install anyio --index-url https://pypi-proxy.fly.dev/basic-auth/simple --strict -v` (exit code: 1)
```
Part of https://github.com/astral-sh/uv/issues/2777
I noticed we're seeing "Python ABI" _a lot_ in error messages which I
did not expect. This improves a common case by being a little more
specific.
Basically, if username-only authentication came from the _cache_ instead
of being present on the _request URL_ to start, we'd end up ignoring it
during password lookups which breaks keyring.
Includes some cosmetic changes to the logging and commentary in the
middleware, because I was confused when reading the code and logs.
The docs reference `UV_INDEX_`, but the code actually uses
UV_HTTP_BASIC_ as the prefix for environment variable credentials.
See PR #7741
Code is at
https://github.com/astral-sh/uv/blob/main/crates/uv-static/src/env_vars.rs#L163
```rust
/// Generates the environment variable key for the HTTP Basic authentication username.
pub fn http_basic_username(name: &str) -> String {
format!("UV_HTTP_BASIC_{name}_USERNAME")
}
/// Generates the environment variable key for the HTTP Basic authentication password.
pub fn http_basic_password(name: &str) -> String {
format!("UV_HTTP_BASIC_{name}_PASSWORD")
}
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
The hack in pip itself only modifies entry points called
`pip<number>.<number>` and `easy_install-<number>.<number>`, uv
previously dropped too many items including any of the form
`foo.<number>`.
Found while trying to install `memray` which somewhat notably does not
provide an abi3 wheel, so the installed, suffixed script matches. At a
minimum, this makes the installed files match the `entry_points.txt`
more than it did previously, which makes `pickley` happy.
## Test Plan
New test provided for previously-untested code.
Closes https://github.com/astral-sh/uv/issues/8228
e.g., on this branch
```
❯ uv python install 3.13t 3.13
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13 --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none
❯ cargo run -q --bin uvx -- -p 3.13t --from build python -c "import sys; print(sys.base_prefix)"
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
and on main
```
❯ cargo build
❯ cargo run -q --bin uvx -- --from build python -c "import sys; print(sys.base_prefix)"
Installed 3 packages in 12ms
/Users/zb/.local/share/uv/python/cpython-3.13.0+freethreaded-macos-aarch64-none
```
I want to add more test coverage around this, but I've noticed the
free-threaded discovery tests are a bit off as-is and it'll be a bigger
task. I think the recent bugs around discovery indicate we should invest
more into that test framework.
Extends #7959
While I was looking at that message, I noticed I didn't love the
readability of the existing message and opted to follow-up with a change
to them both.
When patch version isn't specified and a matching version is referenced,
it will default patch to 0 which could be unclear/confusing. This PR
warns the user of that default.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
The first part of this issue
https://github.com/astral-sh/uv/issues/7426. Will tackle the second part
mentioned (`~=`) in a separate PR once I know this is the correct way to
warn users.
## Test Plan
<!-- How was it tested? -->
Unit tests were added
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
We shouldn't show these in `uv add`, especially when the thing we're
adding is about to have a lower-bound put on it. Now, we only show these
when the user runs `uv lock` or `uv sync`.
## Summary
If you pass a named index via the CLI, you can now reference it as a
named source. This required some surprisingly large refactors, since we
now need to be able to track whether a given index was provided on the
CLI vs. elsewhere (since, e.g., we don't want users to be able to
reference named indexes defined in global configuration).
Closes https://github.com/astral-sh/uv/issues/7899.
## Summary
This PR adds an index pin with `uv add` when the user provides exactly
one named index. We don't pin if the user provides an unnamed index, or
if they provide multiple indexes.
We probably _could_ pin on multiple indexes by writing the sources
_after_ resolution, if that's desirable. But we have no idea which index
the user _expects_ each package to come from.
Possible extensions:
- `uv add --no-pin` to avoid this pinning.
- Warn if they provide a single, unnamed index? I'm not sure if that's
worth a warn. Open to input.
## Summary
This was already properly handled, but the operation itself was in a
`debug_assert!`, so it wasn't running at all in production builds...
Closes https://github.com/astral-sh/uv/issues/8208.
## Summary
This PR lifts the restriction that a package must come from a single
index. For example, you can now do:
```toml
[project]
name = "project"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["jinja2"]
[tool.uv.sources]
jinja2 = [
{ index = "torch-cu118", marker = "sys_platform == 'darwin'"},
{ index = "torch-cu124", marker = "sys_platform != 'darwin'"},
]
[[tool.uv.index]]
name = "torch-cu118"
url = "https://download.pytorch.org/whl/cu118"
[[tool.uv.index]]
name = "torch-cu124"
url = "https://download.pytorch.org/whl/cu124"
```
The construction is very similar to the way we handle URLs today: you
can have multiple URLs for a given package, but they must appear in
disjoint forks. So most of the code is just adding that abstraction to
the resolver, following our handling of URLs.
Closes#7761.
## Summary
The behavior is as follows:
- If you provide `--index` or `--default-index` on the command-line, we
add those indexes to the `pyproject.toml` (with names, if provided, as
in `--index pytorch=https://download.pytorch.org/whl/cu121`.
- If you provide `--index-url` or `--default-index`, we warn, but don't
add the indexes to the file. (This seems wrong -- why not add them?)
- If you provide an index with a name or URL that already exists, we
remove that entry, and add the new index to the top of the list (since
it now has highest priority).
- If you provide a `--default-index`, and an index already has `default
= true`, we remove that entry, since it won't be used anymore.
We do _not_ pin packages to specific indexes yet.
## Summary
This PR enables users to provide index credentials via named environment
variables.
For example, given an index named `internal` that requires a username
(`public`) and password
(`koala`), you can define the index (without credentials) in your
`pyproject.toml`:
```toml
[[tool.uv.index]]
name = "internal"
url = "https://pypi-proxy.corp.dev/simple"
```
Then set the `UV_INDEX_INTERNAL_USERNAME` and
`UV_INDEX_INTERNAL_PASSWORD`
environment variables, where `INTERNAL` is the uppercase version of the
index name:
```sh
export UV_INDEX_INTERNAL_USERNAME=public
export UV_INDEX_INTERNAL_PASSWORD=koala
```
## Summary
This PR adds a first-class API for defining registry indexes, beyond our
existing `--index-url` and `--extra-index-url` setup.
Specifically, you now define indexes like so in a `uv.toml` or
`pyproject.toml` file:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
```
You can also provide indexes via `--index` and `UV_INDEX`, and override
the default index with `--default-index` and `UV_DEFAULT_INDEX`.
### Index priority
Indexes are prioritized in the order in which they're defined, such that
the first-defined index has highest priority.
Indexes are also inherited from parent configuration (e.g., the
user-level `uv.toml`), but are placed after any indexes in the current
project, matching our semantics for other array-based configuration
values.
You can mix `--index` and `--default-index` with the legacy
`--index-url` and `--extra-index-url` settings; the latter two are
merely treated as unnamed `[[tool.uv.index]]` entries.
### Index pinning
If an index includes a name (which is optional), it can then be
referenced via `tool.uv.sources`:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[tool.uv.sources]
torch = { index = "pytorch" }
```
If an index is marked as `explicit = true`, it can _only_ be used via
such references, and will never be searched implicitly:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch" }
```
Indexes defined outside of the current project (e.g., in the user-level
`uv.toml`) can _not_ be explicitly selected.
(As of now, we only support using a single index for a given
`tool.uv.sources` definition.)
### Default index
By default, we include PyPI as the default index. This remains true even
if the user defines a `[[tool.uv.index]]` -- PyPI is still used as a
fallback. You can mark an index as `default = true` to (1) disable the
use of PyPI, and (2) bump it to the bottom of the prioritized list, such
that it's used only if a package does not exist on a prior index:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
default = true
```
### Name reuse
If a name is reused, the higher-priority index with that name is used,
while the lower-priority indexes are ignored entirely.
For example, given:
```toml
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu121"
[[tool.uv.index]]
name = "pytorch"
url = "https://test.pypi.org/simple"
```
The `https://test.pypi.org/simple` index would be ignored entirely,
since it's lower-priority than `https://download.pytorch.org/whl/cu121`
but shares the same name.
Closes#171.
## Future work
- Users should be able to provide authentication for named indexes via
environment variables.
- `uv add` should automatically write `--index` entries to the
`pyproject.toml` file.
- Users should be able to provide multiple indexes for a given package,
stratified by platform:
```toml
[tool.uv.sources]
torch = [
{ index = "cpu", markers = "sys_platform == 'darwin'" },
{ index = "gpu", markers = "sys_platform != 'darwin'" },
]
```
- Users should be able to specify a proxy URL for a given index, to
avoid writing user-specific URLs to a lockfile:
```toml
[[tool.uv.index]]
name = "test"
url = "https://private.org/simple"
proxy = "http://<omitted>/pypi/simple"
```
## Summary
Cache the path to git executable in a `LazyLock` and reuse it throughout
the process. This might reduce some costs on finding the git executable.
When building a source distribution to a wheels, we perform the build
inside a temporary directory inside the output directory. By default,
the output directory is `dist/` in the repository root. This temp dir
placement allows us to move the final wheel to the output directory
instead of copying it (a temp dir might be on another device, which
means we need to copy instead of moving).
Some build backends such as hatchling traverse upwards from the current
directory (the source dist build location) looking for gitignore files
to consider. By adding a gitignore in `dist/` with `*`, we caused
hatchling to ignore all files in our temporary build directory below it,
causing empty wheels. To prevent this, we add a `.git` file as a phony
git root. We are already using this trick successfully in the cache.
Hatchling sees this `.git` file, considers it a boundary and does not
traverse up to `dist/.gitignore`.
Fixes#8200
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Closes https://github.com/astral-sh/uv/issues/8213
I didn't mean to remove these when updating the regular expression.
Arguably, they shouldn't be used anymore, but we should make that choice
with intention.
As mentioned in https://github.com/astral-sh/uv/issues/8189
We only checked if an interpreter was free-threaded _when_ free-threaded
variants were requested. But we should not use free-threaded
interpreters unless explicitly requested.
## Summary
This PR declares and documents all environment variables that are used
in one way or another in `uv`, either internally, or externally, or
transitively under a common struct.
I think over time as uv has grown there's been many environment
variables introduced. Its harder to know which ones exists, which ones
are missing, what they're used for, or where are they used across the
code. The docs only documents a handful of them, for others you'd have
to dive into the code and inspect across crates to know which crates
they're used on or where they're relevant.
This PR is a starting attempt to unify them, make it easier to discover
which ones we have, and maybe unlock future posibilities in automating
generating documentation for them.
I think we can split out into multiple structs later to better organize,
but given the high influx of PR's and possibly new environment variables
introduced/re-used, it would be hard to try to organize them all now
into their proper namespaced struct while this is all happening given
merge conflicts and/or keeping up to date.
I don't think this has any impact on performance as they all should
still be inlined, although it may affect local build times on changes to
the environment vars as more crates would likely need a rebuild. Lastly,
some of them are declared but not used in the code, for example those in
`build.rs`. I left them declared because I still think it's useful to at
least have a reference.
Did I miss any? Are their initial docs cohesive?
Note, `uv-static` is a terrible name for a new crate, thoughts? Others
considered `uv-vars`, `uv-consts`.
## Test Plan
Existing tests
Charlie Marsh
Andrew Gallant
Zanie Blue
konsti
Pietro Zambelli
Jo
Simon Willison
renovate[bot]
Tim Chan
Vivien Maisonneuve
Magnus Markling
Nathan McDougall
Andrey Bozhko
tfsingh
Ahmed Ilyas
Amjith Ramanujam
kkdamowang
Alexander Gherm
InSync
Vini Brasil
David Bern
Tim Hatch
github-actions[bot]
Aditya Pratap Singh
samypr100
Ian Paul