Python
/
uv
mirror of https://github.com/astral-sh/uv
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add a `--platform` argument to enable resolving against a target platform (#3111) 

## Summary

I've wanted to try this for a long time, so decided to give it a shot.
The basic idea is that you can provide a target triple (e.g.,
`--platform x86_64-pc-windows-msvc`) and resolve against that platform,
rather than the currently-running platform. It's functionally similar to
`--python-version`, though a bit simpler since there's no need to engage
with `Requires-Python`.

Our infrastructure is well-setup for this and so, in the end, it's
actually pretty straightforward: for each triple, we just need to
override the markers and platform tags.
This commit is contained in:
Charlie Marsh 2024-04-18 22:57:41 -04:00 committed by GitHub
parent 9259eceebc
commit 93559d5c2a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 328 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
README.md
View File

@ -384,16 +384,32 @@ While constraints are purely _additive_, and thus cannot _expand_ the set of acc
a package, overrides _can_ expand the set of acceptable versions for a package, providing an escape
hatch for erroneous upper version bounds.
### Multi-version resolution
### Multi-platform resolution
uv's `pip-compile` command produces a resolution that's known to be compatible with the
current platform and Python version. Unlike Poetry, PDM, and other package managers, uv does
not yet produce a machine-agnostic lockfile.
By default, uv's `pip-compile` command produces a resolution that's known to be compatible with
the current platform and Python version. Unlike Poetry and PDM, uv does not yet produce a
machine-agnostic lockfile ([#2679](https://github.com/astral-sh/uv/issues/2679)).
However, uv _does_ support resolving for alternate Python versions via the `--python-version`
command line argument. For example, if you're running uv on Python 3.9, but want to resolve for
Python 3.8, you can run `uv pip compile --python-version=3.8 requirements.in` to produce a
Python 3.8-compatible resolution.
However, uv _does_ support resolving for alternate platforms and Python versions via the
`--platform` and `--python-version` command line arguments.
For example, if you're running uv on macOS, but want to resolve for Linux, you can run
`uv pip compile --platform=linux requirements.in` to produce a `manylinux2014`-compatible
resolution.
Similarly, if you're running uv on Python 3.9, but want to resolve for Python 3.8, you can run
`uv pip compile --python-version=3.8 requirements.in` to produce a Python 3.8-compatible resolution.
The `--platform` and `--python-version` arguments can be combined to produce a resolution for
a specific platform and Python version, enabling users to generate multiple lockfiles for
different environments from a single machine.
_N.B. Python's environment markers expose far more information about the current machine
than can be expressed by a simple `--platform` argument. For example, the `platform_version` marker
on macOS includes the time at which the kernel was built, which can (in theory) be encoded in
package requirements. uv's resolver makes a best-effort attempt to generate a resolution that is
compatible with any machine running on the target `--platform`, which should be sufficient for
most use cases, but may lose fidelity for complex package and platform combinations._
### Reproducible resolution

9
crates/uv/src/cli.rs
View File

@ -16,6 +16,7 @@ use uv_toolchain::PythonVersion;
use crate::commands::{extra_name_with_clap_error, ListFormat, VersionFormat};
use crate::compat;
use crate::target::TargetTriple;
#[derive(Parser)]
#[command(author, version, long_version = crate::version::version(), about)]
@ -520,6 +521,14 @@ pub(crate) struct PipCompileArgs {
#[arg(long, short)]
pub(crate) python_version: Option<PythonVersion>,
/// The platform for which requirements should be resolved.
///
/// Represented as a "target triple", a string that describes the target platform in terms of
/// its CPU, vendor, and operating system name, like `x86_64-unknown-linux-gnu` or
/// `aaarch64-apple-darwin`.
#[arg(long)]
pub(crate) platform: Option<TargetTriple>,
/// Limit candidate packages to those that were uploaded prior to the given date.
///
/// Accepts both RFC 3339 timestamps (e.g., `2006-12-02T02:07:43Z`) and UTC dates in the same

39
crates/uv/src/commands/pip_compile.rs
View File

@ -46,6 +46,7 @@ use uv_warnings::warn_user;
use crate::commands::reporters::{DownloadReporter, ResolverReporter};
use crate::commands::{elapsed, ExitStatus};
use crate::printer::Printer;
use crate::target::TargetTriple;
/// Resolve a set of requirements into a set of pinned versions.
#[allow(clippy::too_many_arguments, clippy::fn_params_excessive_bools)]
@ -78,6 +79,7 @@ pub(crate) async fn pip_compile(
no_build_isolation: bool,
no_build: NoBuild,
python_version: Option<PythonVersion>,
target: Option<TargetTriple>,
exclude_newer: Option<ExcludeNewer>,
annotation_style: AnnotationStyle,
link_mode: LinkMode,
@ -193,21 +195,40 @@ pub(crate) async fn pip_compile(
};
// Determine the tags, markers, and interpreter to use for resolution.
let tags = if let Some(python_version) = python_version.as_ref() {
Cow::Owned(Tags::from_env(
let tags = match (target, python_version.as_ref()) {
(Some(target), Some(python_version)) => Cow::Owned(Tags::from_env(
&target.platform(),
(python_version.major(), python_version.minor()),
interpreter.implementation_name(),
interpreter.implementation_tuple(),
interpreter.gil_disabled(),
)?),
(Some(target), None) => Cow::Owned(Tags::from_env(
&target.platform(),
interpreter.python_tuple(),
interpreter.implementation_name(),
interpreter.implementation_tuple(),
interpreter.gil_disabled(),
)?),
(None, Some(python_version)) => Cow::Owned(Tags::from_env(
interpreter.platform(),
(python_version.major(), python_version.minor()),
interpreter.implementation_name(),
interpreter.implementation_tuple(),
interpreter.gil_disabled(),
)?)
} else {
Cow::Borrowed(interpreter.tags()?)
)?),
(None, None) => Cow::Borrowed(interpreter.tags()?),
};
// Apply the platform tags to the markers.
let markers = match (target, python_version) {
(Some(target), Some(python_version)) => {
Cow::Owned(python_version.markers(&target.markers(interpreter.markers())))
}
(Some(target), None) => Cow::Owned(target.markers(interpreter.markers())),
(None, Some(python_version)) => Cow::Owned(python_version.markers(interpreter.markers())),
(None, None) => Cow::Borrowed(interpreter.markers()),
};
let markers = python_version.map_or_else(
|| Cow::Borrowed(interpreter.markers()),
|python_version| Cow::Owned(python_version.markers(interpreter.markers())),
);
// Generate, but don't enforce hashes for the requirements.
let hasher = if generate_hashes {

2
crates/uv/src/main.rs
View File

@ -50,6 +50,7 @@ mod logging;
mod printer;
mod settings;
mod shell;
mod target;
mod version;
#[instrument]
@ -254,6 +255,7 @@ async fn run() -> Result<ExitStatus> {
args.shared.no_build_isolation,
no_build,
args.shared.python_version,
args.platform,
args.shared.exclude_newer,
args.shared.annotation_style,
args.shared.link_mode,

4
crates/uv/src/settings.rs
View File

@ -14,6 +14,7 @@ use crate::cli::{
PipListArgs, PipShowArgs, PipSyncArgs, PipUninstallArgs, VenvArgs,
};
use crate::commands::ListFormat;
use crate::target::TargetTriple;
/// The resolved global settings to use for any invocation of the CLI.
#[allow(clippy::struct_excessive_bools)]
@ -74,6 +75,7 @@ pub(crate) struct PipCompileSettings {
pub(crate) src_file: Vec<PathBuf>,
pub(crate) constraint: Vec<PathBuf>,
pub(crate) r#override: Vec<PathBuf>,
pub(crate) platform: Option<TargetTriple>,
pub(crate) refresh: bool,
pub(crate) refresh_package: Vec<PackageName>,
pub(crate) upgrade: bool,
@ -134,6 +136,7 @@ impl PipCompileSettings {
only_binary,
config_setting,
python_version,
platform,
exclude_newer,
no_emit_package,
emit_index_url,
@ -152,6 +155,7 @@ impl PipCompileSettings {
src_file,
constraint,
r#override,
platform,
refresh,
refresh_package: refresh_package.unwrap_or_default(),
upgrade,

190
crates/uv/src/target.rs Normal file
View File

@ -0,0 +1,190 @@
use pep508_rs::MarkerEnvironment;
use platform_tags::{Arch, Os, Platform};
/// The supported target triples. Each triple consists of an architecture, vendor, and operating
/// system.
///
/// See: <https://doc.rust-lang.org/nightly/rustc/platform-support.html>
#[derive(Debug, Clone, Copy, Eq, PartialEq, clap::ValueEnum)]
pub(crate) enum TargetTriple {
/// An alias for `x86_64-pc-windows-msvc`, the default target for Windows.
Windows,
/// An alias for `x86_64-unknown-linux-gnu`, the default target for Linux.
Linux,
/// An alias for `aarch64-apple-darwin`, the default target for macOS.
Macos,
/// An x86 Windows target.
#[value(name = "x86_64-pc-windows-msvc")]
X8664PcWindowsMsvc,
/// An x86 Linux target.
#[value(name = "x86_64-unknown-linux-gnu")]
X8664UnknownLinuxGnu,
/// An ARM-based macOS target, as seen on Apple Silicon devices.
#[value(name = "aarch64-apple-darwin")]
Aarch64AppleDarwin,
/// An x86 macOS target.
#[value(name = "x86_64-apple-darwin")]
X8664AppleDarwin,
/// An ARM64 Linux target.
#[value(name = "aarch64-unknown-linux-gnu")]
Aarch64UnknownLinuxGnu,
/// An ARM64 Linux target.
#[value(name = "aarch64-unknown-linux-musl")]
Aarch64UnknownLinuxMusl,
/// An x86_64 Linux target.
#[value(name = "x86_64-unknown-linux-musl")]
X8664UnknownLinuxMusl,
}
impl TargetTriple {
/// Return the [`Platform`] for the target.
pub(crate) fn platform(self) -> Platform {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => Platform::new(Os::Windows, Arch::X86_64),
Self::Linux | Self::X8664UnknownLinuxGnu => Platform::new(
Os::Manylinux {
major: 2,
minor: 17,
},
Arch::X86_64,
),
Self::Macos | Self::Aarch64AppleDarwin => Platform::new(
Os::Macos {
major: 11,
minor: 0,
},
Arch::Aarch64,
),
Self::X8664AppleDarwin => Platform::new(
Os::Macos {
major: 10,
minor: 12,
},
Arch::X86_64,
),
Self::Aarch64UnknownLinuxGnu => Platform::new(
Os::Manylinux {
major: 2,
minor: 17,
},
Arch::Aarch64,
),
Self::Aarch64UnknownLinuxMusl => {
Platform::new(Os::Musllinux { major: 1, minor: 2 }, Arch::Aarch64)
}
Self::X8664UnknownLinuxMusl => {
Platform::new(Os::Musllinux { major: 1, minor: 2 }, Arch::X86_64)
}
}
}
/// Return the `platform_machine` value for the target.
pub(crate) fn platform_machine(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "x86_64",
Self::Linux | Self::X8664UnknownLinuxGnu => "x86_64",
Self::Macos | Self::Aarch64AppleDarwin => "aarch64",
Self::X8664AppleDarwin => "x86_64",
Self::Aarch64UnknownLinuxGnu => "aarch64",
Self::Aarch64UnknownLinuxMusl => "aarch64",
Self::X8664UnknownLinuxMusl => "x86_64",
}
}
/// Return the `platform_system` value for the target.
pub(crate) fn platform_system(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "Windows",
Self::Linux | Self::X8664UnknownLinuxGnu => "Linux",
Self::Macos | Self::Aarch64AppleDarwin => "Darwin",
Self::X8664AppleDarwin => "Darwin",
Self::Aarch64UnknownLinuxGnu => "Linux",
Self::Aarch64UnknownLinuxMusl => "Linux",
Self::X8664UnknownLinuxMusl => "Linux",
}
}
/// Return the `platform_version` value for the target.
pub(crate) fn platform_version(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "",
Self::Linux | Self::X8664UnknownLinuxGnu => "",
Self::Macos | Self::Aarch64AppleDarwin => "",
Self::X8664AppleDarwin => "",
Self::Aarch64UnknownLinuxGnu => "",
Self::Aarch64UnknownLinuxMusl => "",
Self::X8664UnknownLinuxMusl => "",
}
}
/// Return the `platform_release` value for the target.
pub(crate) fn platform_release(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "",
Self::Linux | Self::X8664UnknownLinuxGnu => "",
Self::Macos | Self::Aarch64AppleDarwin => "",
Self::X8664AppleDarwin => "",
Self::Aarch64UnknownLinuxGnu => "",
Self::Aarch64UnknownLinuxMusl => "",
Self::X8664UnknownLinuxMusl => "",
}
}
/// Return the `os_name` value for the target.
pub(crate) fn os_name(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "nt",
Self::Linux | Self::X8664UnknownLinuxGnu => "posix",
Self::Macos | Self::Aarch64AppleDarwin => "posix",
Self::X8664AppleDarwin => "posix",
Self::Aarch64UnknownLinuxGnu => "posix",
Self::Aarch64UnknownLinuxMusl => "posix",
Self::X8664UnknownLinuxMusl => "posix",
}
}
/// Return the `sys_platform` value for the target.
pub(crate) fn sys_platform(self) -> &'static str {
match self {
Self::Windows | Self::X8664PcWindowsMsvc => "win32",
Self::Linux | Self::X8664UnknownLinuxGnu => "linux",
Self::Macos | Self::Aarch64AppleDarwin => "darwin",
Self::X8664AppleDarwin => "darwin",
Self::Aarch64UnknownLinuxGnu => "linux",
Self::Aarch64UnknownLinuxMusl => "linux",
Self::X8664UnknownLinuxMusl => "linux",
}
}
/// Return a [`MarkerEnvironment`] compatible with the given [`TargetTriple`], based on
/// a base [`MarkerEnvironment`].
///
/// The returned [`MarkerEnvironment`] will preserve the base environment's Python version
/// markers, but override its platform markers.
pub(crate) fn markers(self, base: &MarkerEnvironment) -> MarkerEnvironment {
MarkerEnvironment {
// Platform markers
os_name: self.os_name().to_string(),
platform_machine: self.platform_machine().to_string(),
platform_system: self.platform_system().to_string(),
sys_platform: self.sys_platform().to_string(),
platform_release: self.platform_release().to_string(),
platform_version: self.platform_version().to_string(),
// Python version markers
implementation_name: base.implementation_name.clone(),
implementation_version: base.implementation_version.clone(),
platform_python_implementation: base.platform_python_implementation.clone(),
python_full_version: base.python_full_version.clone(),
python_version: base.python_version.clone(),
}
}
}

69
crates/uv/tests/pip_compile.rs
View File

@ -7890,6 +7890,75 @@ fn no_version_for_direct_dependency() -> Result<()> {
Ok(())
}
/// Compile against a dedicated platform, which may differ from the current platform.
#[test]
fn platform() -> Result<()> {
let context = TestContext::new("3.12");
let requirements_in = context.temp_dir.child("requirements.in");
requirements_in.write_str("black")?;
uv_snapshot!(context.filters(),
windows_filters=false,
context.compile()
.arg("requirements.in")
.arg("--platform")
.arg("aarch64-unknown-linux-gnu"), @r###"
success: true
exit_code: 0
----- stdout -----
# This file was autogenerated by uv via the following command:
# uv pip compile --cache-dir [CACHE_DIR] --exclude-newer 2024-03-25T00:00:00Z requirements.in --platform aarch64-unknown-linux-gnu
black==24.3.0
click==8.1.7
# via black
mypy-extensions==1.0.0
# via black
packaging==24.0
# via black
pathspec==0.12.1
# via black
platformdirs==4.2.0
# via black
----- stderr -----
Resolved 6 packages in [TIME]
"###
);
uv_snapshot!(context.filters(),
windows_filters=false,
context.compile()
.arg("requirements.in")
.arg("--platform")
.arg("x86_64-pc-windows-msvc"), @r###"
success: true
exit_code: 0
----- stdout -----
# This file was autogenerated by uv via the following command:
# uv pip compile --cache-dir [CACHE_DIR] --exclude-newer 2024-03-25T00:00:00Z requirements.in --platform x86_64-pc-windows-msvc
black==24.3.0
click==8.1.7
# via black
colorama==0.4.6
# via click
mypy-extensions==1.0.0
# via black
packaging==24.0
# via black
pathspec==0.12.1
# via black
platformdirs==4.2.0
# via black
----- stderr -----
Resolved 7 packages in [TIME]
"###
);
Ok(())
}
/// Verify that command-line arguments take precedence over on-disk configuration.
#[test]
fn resolve_configuration() -> Result<()> {