Change backtracking when packages conflict too much (#9843)

Background reading: https://github.com/astral-sh/uv/issues/8157
Companion PR: https://github.com/astral-sh/pubgrub/pull/36
Requires for test coverage: https://github.com/astral-sh/packse/pull/230

When two packages A and B conflict, we have the option to choose a lower
version of A, or a lower version of B. Currently, we determine this by
the order we saw a package (assuming equal specificity of the
requirement): If we saw A before B, we pin A until all versions of B are
exhausted. This can lead to undesirable outcomes, from cases where it's
just slow (sentry) to others cases without lower bounds where be
backtrack to a very old version of B. This old version may fail to build
(terminating the resolution), or it's a version so old that it doesn't
depend on A (or the shared conflicting package) anymore - but also is
too old for the user's application (fastapi). #8157 collects such cases,
and the `wrong-backtracking` packse scenario contains a minimized
example.

We try to solve this by tracking which packages are "A"s, culprits, and
"B"s, affected, and manually interfering with project selection and
backtracking. Whenever a version we just chose is rejected, we give the
current package a counter for being affected, and the package it
conflicted with a counter for being a culprit. If a package accumulates
more counts than a threshold, we reprioritize: Undecided after the
culprits, after the affected, after packages that only have a single
version (URLs, `==<version>`). We then ask pubgrub to backtrack just
before the culprit. Due to the changed priorities, we now select package
B, the affected, instead of package A, the culprit.

To do this efficiently, we ask pubgrub for the incompatibility that
caused backtracking, or just the last version to be discarded (due to
its dependencies). For backtracking, we use the last incompatibility
from unit propagation as a heuristic. When a version is discarded
because one of its dependencies conflicts with the partial solution, the
incompatibility tells us the package in the partial solution that
conflicted.

We only backtrack once per package, on the first time it passes the
threshold. This prevents backtracking loops in which we make the same
decisions over and over again. But we also changed the priority, so that
we shouldn't take the same path even after the one time we backtrack (it
would defeat the purpose of this change).

There are some parameters that can be tweaked: Currently, the threshold
is set to 5, which feels not too eager with so me of the conflicts that
we want to tolerate but also changes strategies quickly. The relative
order of the new priorities can also be changed, as for each (A, B) pair
the priority of B is afterwards lower than that for A. Currently,
culprits capture conflict for the whole package, but we could limit that
to a specific version. We could discard conflict counters after
backtracking instead of keeping them eternally as we do now. Note that
we're always taking about pairs (A, B), but in practice we track
individual packages, not pairs.

A case that we wouldn't capture is when B is only introduced to the
dependency graph after A, but I think that would require cyclical
dependency for A and B to conflict? There may also be cases where
looking at the last incompatibility is insufficient.

Another example that we can't repair with prioritization is
urllib3/boto3/botocore: We actually have to check all the newer versions
of boto3 and botocore to identify the version that allows with the older
urllib3, no shortcuts allowed.

```
urllib3<1.25.4
boto3
```

All examples I tested were cases with two packages where we only had to
switch the order, so I've abstracted them into a single packse case.

This PR changes the resolution for certain paths, and there is the risk
for regressions.

Fixes #8157

---

All tested examples improved.

Input fastapi:
```text
starlette<=0.36.0
fastapi<=0.115.2
```

```
# BEFORE
$ uv pip --no-progress compile -p 3.11 --exclude-newer 2024-10-01 --no-annotate debug/fastapi.txt
annotated-types==0.7.0
anyio==4.6.0
fastapi==0.1.17
idna==3.10
pydantic==2.9.2
pydantic-core==2.23.4
sniffio==1.3.1
starlette==0.36.0
typing-extensions==4.12.2

# AFTER
$ cargo run --profile fast-build --no-default-features pip compile -p 3.11 --no-progress --exclude-newer 2024-10-01 --no-annotate debug/fastapi.txt 
annotated-types==0.7.0
anyio==4.6.0
fastapi==0.109.1
idna==3.10
pydantic==2.9.2
pydantic-core==2.23.4
sniffio==1.3.1
starlette==0.35.1
typing-extensions==4.12.2
```


Input xarray:
```text
xarray[accel]
```

```
# BEFORE
$ uv pip --no-progress compile -p 3.11 --exclude-newer 2024-10-01 --no-annotate debug/xarray-accel.txt
bottleneck==1.4.0
flox==0.9.13
llvmlite==0.36.0
numba==0.53.1
numbagg==0.8.2
numpy==2.1.1
numpy-groupies==0.11.2
opt-einsum==3.4.0
packaging==24.1
pandas==2.2.3
python-dateutil==2.9.0.post0
pytz==2024.2
scipy==1.14.1
setuptools==75.1.0
six==1.16.0
toolz==0.12.1
tzdata==2024.2
xarray==2024.9.0

# AFTER
$ cargo run --profile fast-build --no-default-features pip compile -p 3.11 --no-progress --exclude-newer 2024-10-01 --no-annotate debug/xarray-accel.txt
bottleneck==1.4.0
flox==0.9.13
llvmlite==0.43.0
numba==0.60.0
numbagg==0.8.2
numpy==2.0.2
numpy-groupies==0.11.2
opt-einsum==3.4.0
packaging==24.1
pandas==2.2.3
python-dateutil==2.9.0.post0
pytz==2024.2
scipy==1.14.1
six==1.16.0
toolz==0.12.1
tzdata==2024.2
xarray==2024.9.0
```


Input sentry: The resolution is identical, but arrived at much faster:
main tries 69 versions (sentry-kafka-schemas: 63), PR tries 12 versions
(sentry-kafka-schemas: 6; 5 times conflicting, then once the right
version).

```text
python-rapidjson<=1.20,>=1.4
sentry-kafka-schemas<=0.1.113,>=0.1.50
```

```
# BEFORE
$ uv pip --no-progress compile -p 3.11 --exclude-newer 2024-10-01 --no-annotate debug/sentry.txt
fastjsonschema==2.20.0
msgpack==1.1.0
python-rapidjson==1.8
pyyaml==6.0.2
sentry-kafka-schemas==0.1.111
typing-extensions==4.12.2

# AFTER
$ cargo run --profile fast-build --no-default-features pip compile -p 3.11 --no-progress --exclude-newer 2024-10-01 --no-annotate debug/sentry.txt
fastjsonschema==2.20.0
msgpack==1.1.0
python-rapidjson==1.8
pyyaml==6.0.2
sentry-kafka-schemas==0.1.111
typing-extensions==4.12.2
```


Input apache-beam
```text
# Run on Python 3.10
dill<0.3.9,>=0.2.2
apache-beam<=2.49.0
```

```
# BEFORE
$ uv pip --no-progress compile -p 3.10 --exclude-newer 2024-10-01 --no-annotate debug/apache-beam.txt
  × Failed to download and build `apache-beam==2.0.0`
  ╰─▶ Build backend failed to determine requirements with `build_wheel()` (exit status: 1)

# AFTER
$ cargo run --profile fast-build --no-default-features pip compile -p 3.10 --no-progress --exclude-newer 2024-10-01 --no-annotate debug/apache-beam.txt
apache-beam==2.49.0
certifi==2024.8.30
charset-normalizer==3.3.2
cloudpickle==2.2.1
crcmod==1.7
dill==0.3.1.1
dnspython==2.6.1
docopt==0.6.2
fastavro==1.9.7
fasteners==0.19
grpcio==1.66.2
hdfs==2.7.3
httplib2==0.22.0
idna==3.10
numpy==1.24.4
objsize==0.6.1
orjson==3.10.7
proto-plus==1.24.0
protobuf==4.23.4
pyarrow==11.0.0
pydot==1.4.2
pymongo==4.10.0
pyparsing==3.1.4
python-dateutil==2.9.0.post0
pytz==2024.2
regex==2024.9.11
requests==2.32.3
six==1.16.0
typing-extensions==4.12.2
urllib3==2.2.3
zstandard==0.23.0
```
This commit is contained in:
konsti 2024-12-16 11:39:50 +01:00 committed by GitHub
parent 2b61a67cf7
commit 431ddc1d74
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 332 additions and 54 deletions

4
Cargo.lock generated
View File

@ -2673,7 +2673,7 @@ dependencies = [
[[package]]
name = "pubgrub"
version = "0.2.1"
source = "git+https://github.com/astral-sh/pubgrub?rev=57832d0588fbb7aab824813481104761dc1c7740#57832d0588fbb7aab824813481104761dc1c7740"
source = "git+https://github.com/astral-sh/pubgrub?rev=05e8d12cea8d72c6d2d017900e478d0abd28fef4#05e8d12cea8d72c6d2d017900e478d0abd28fef4"
dependencies = [
"indexmap",
"log",
@ -5672,7 +5672,7 @@ checksum = "830b7e5d4d90034032940e4ace0d9a9a057e7a45cd94e6c007832e39edb82f6d"
[[package]]
name = "version-ranges"
version = "0.1.1"
source = "git+https://github.com/astral-sh/pubgrub?rev=57832d0588fbb7aab824813481104761dc1c7740#57832d0588fbb7aab824813481104761dc1c7740"
source = "git+https://github.com/astral-sh/pubgrub?rev=05e8d12cea8d72c6d2d017900e478d0abd28fef4#05e8d12cea8d72c6d2d017900e478d0abd28fef4"
dependencies = [
"smallvec",
]

View File

@ -130,7 +130,7 @@ petgraph = { version = "0.6.5" }
platform-info = { version = "2.0.3" }
proc-macro2 = { version = "1.0.86" }
procfs = { version = "0.17.0", default-features = false, features = ["flate2"] }
pubgrub = { git = "https://github.com/astral-sh/pubgrub", rev = "57832d0588fbb7aab824813481104761dc1c7740" }
pubgrub = { git = "https://github.com/astral-sh/pubgrub", rev = "05e8d12cea8d72c6d2d017900e478d0abd28fef4" }
quote = { version = "1.0.37" }
rayon = { version = "1.10.0" }
reflink-copy = { version = "0.1.19" }
@ -175,7 +175,7 @@ unicode-width = { version = "0.1.13" }
unscanny = { version = "0.1.0" }
url = { version = "2.5.2", features = ["serde"] }
urlencoding = { version = "2.1.3" }
version-ranges = { git = "https://github.com/astral-sh/pubgrub", rev = "57832d0588fbb7aab824813481104761dc1c7740" }
version-ranges = { git = "https://github.com/astral-sh/pubgrub", rev = "05e8d12cea8d72c6d2d017900e478d0abd28fef4" }
walkdir = { version = "2.5.0" }
which = { version = "7.0.0", features = ["regex"] }
windows-registry = { version = "0.3.0" }

View File

@ -1,6 +1,6 @@
use std::convert::Infallible;
use pubgrub::{Dependencies, DependencyProvider, Range};
use pubgrub::{Dependencies, DependencyProvider, PackageResolutionStatistics, Range};
use uv_pep440::Version;
@ -17,13 +17,17 @@ impl DependencyProvider for UvDependencyProvider {
type V = Version;
type VS = Range<Version>;
type M = UnavailableReason;
type Priority = Option<PubGrubPriority>;
type Err = Infallible;
fn prioritize(&self, _package: &Self::P, _range: &Self::VS) -> Self::Priority {
fn prioritize(
&self,
_package: &Self::P,
_range: &Self::VS,
_stats: &PackageResolutionStatistics,
) -> Self::Priority {
unimplemented!()
}
type Priority = Option<PubGrubPriority>;
type Err = Infallible;
fn choose_version(
&self,

View File

@ -1,7 +1,7 @@
use std::cmp::Reverse;
use pubgrub::Range;
use rustc_hash::FxHashMap;
use std::cmp::Reverse;
use std::collections::hash_map::OccupiedEntry;
use crate::fork_urls::ForkUrls;
use uv_normalize::PackageName;
@ -40,12 +40,7 @@ impl PubGrubPriorities {
match self.0.entry(name.clone()) {
std::collections::hash_map::Entry::Occupied(mut entry) => {
// Preserve the original index.
let index = match entry.get() {
PubGrubPriority::Unspecified(Reverse(index)) => *index,
PubGrubPriority::Singleton(Reverse(index)) => *index,
PubGrubPriority::DirectUrl(Reverse(index)) => *index,
PubGrubPriority::Root => next,
};
let index = Self::get_index(&entry).unwrap_or(next);
// Compute the priority.
let priority = if urls.get(name).is_some() {
@ -53,6 +48,14 @@ impl PubGrubPriorities {
} else if version.as_singleton().is_some() {
PubGrubPriority::Singleton(Reverse(index))
} else {
// Keep the conflict-causing packages to avoid loops where we seesaw between
// `Unspecified` and `Conflict*`.
if matches!(
entry.get(),
PubGrubPriority::ConflictEarly(_) | PubGrubPriority::ConflictLate(_)
) {
return;
}
PubGrubPriority::Unspecified(Reverse(index))
};
@ -77,6 +80,17 @@ impl PubGrubPriorities {
}
}
fn get_index(entry: &OccupiedEntry<PackageName, PubGrubPriority>) -> Option<usize> {
match entry.get() {
PubGrubPriority::ConflictLate(Reverse(index))
| PubGrubPriority::Unspecified(Reverse(index))
| PubGrubPriority::ConflictEarly(Reverse(index))
| PubGrubPriority::Singleton(Reverse(index))
| PubGrubPriority::DirectUrl(Reverse(index)) => Some(*index),
PubGrubPriority::Root => None,
}
}
/// Return the [`PubGrubPriority`] of the given package, if it exists.
pub(crate) fn get(&self, package: &PubGrubPackage) -> Option<PubGrubPriority> {
match &**package {
@ -88,6 +102,79 @@ impl PubGrubPriorities {
PubGrubPackageInner::Package { name, .. } => self.0.get(name).copied(),
}
}
/// Mark a package as prioritized by setting it to [`PubGrubPriority::ConflictEarly`], if it
/// doesn't have a higher priority already.
///
/// Returns whether the priority was changed, i.e., it's the first time we hit this condition
/// for the package.
pub(crate) fn mark_conflict_early(&mut self, package: &PubGrubPackage) -> bool {
let next = self.0.len();
let Some(name) = package.name_no_root() else {
// Not a correctness bug
if cfg!(debug_assertions) {
panic!("URL packages must not be involved in conflict handling")
} else {
return false;
}
};
match self.0.entry(name.clone()) {
std::collections::hash_map::Entry::Occupied(mut entry) => {
if matches!(
entry.get(),
PubGrubPriority::ConflictEarly(_) | PubGrubPriority::Singleton(_)
) {
// Already in the right category
return false;
};
let index = Self::get_index(&entry).unwrap_or(next);
entry.insert(PubGrubPriority::ConflictEarly(Reverse(index)));
true
}
std::collections::hash_map::Entry::Vacant(entry) => {
entry.insert(PubGrubPriority::ConflictEarly(Reverse(next)));
true
}
}
}
/// Mark a package as prioritized by setting it to [`PubGrubPriority::ConflictLate`], if it
/// doesn't have a higher priority already.
///
/// Returns whether the priority was changed, i.e., it's the first time this package was
/// marked as conflicting above the threshold.
pub(crate) fn mark_conflict_late(&mut self, package: &PubGrubPackage) -> bool {
let next = self.0.len();
let Some(name) = package.name_no_root() else {
// Not a correctness bug
if cfg!(debug_assertions) {
panic!("URL packages must not be involved in conflict handling")
} else {
return false;
}
};
match self.0.entry(name.clone()) {
std::collections::hash_map::Entry::Occupied(mut entry) => {
// The ConflictEarly` match avoids infinite loops.
if matches!(
entry.get(),
PubGrubPriority::ConflictLate(_)
| PubGrubPriority::ConflictEarly(_)
| PubGrubPriority::Singleton(_)
) {
// Already in the right category
return false;
};
let index = Self::get_index(&entry).unwrap_or(next);
entry.insert(PubGrubPriority::ConflictLate(Reverse(index)));
true
}
std::collections::hash_map::Entry::Vacant(entry) => {
entry.insert(PubGrubPriority::ConflictLate(Reverse(next)));
true
}
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
@ -101,6 +188,15 @@ pub(crate) enum PubGrubPriority {
/// in the dependency graph.
Unspecified(Reverse<usize>),
/// Selected version of this package were often the culprit of rejecting another package, so
/// it's deprioritized behind `ConflictEarly`. It's still the higher than `Unspecified` to
/// conflict before selecting unrelated packages.
ConflictLate(Reverse<usize>),
/// Selected version of this package were often rejected, so it's prioritized over
/// `ConflictLate`.
ConflictEarly(Reverse<usize>),
/// The version range is constrained to a single version (e.g., with the `==` operator).
Singleton(Reverse<usize>),

View File

@ -13,7 +13,7 @@ use dashmap::DashMap;
use either::Either;
use futures::{FutureExt, StreamExt};
use itertools::Itertools;
use pubgrub::{Id, Incompatibility, Range, State};
use pubgrub::{Id, IncompId, Incompatibility, Range, Ranges, State};
use rustc_hash::{FxHashMap, FxHashSet};
use tokio::sync::mpsc::{self, Receiver, Sender};
use tokio::sync::oneshot;
@ -89,6 +89,9 @@ mod provider;
mod reporter;
mod urls;
/// The number of conflicts a package may accumulate before we re-prioritize and backtrack.
const CONFLICT_THRESHOLD: usize = 5;
pub struct Resolver<Provider: ResolverProvider, InstalledPackages: InstalledPackagesProvider> {
state: ResolverState<InstalledPackages>,
provider: Provider,
@ -332,16 +335,27 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
initial
} else {
// Run unit propagation.
if let Err(err) = state.pubgrub.unit_propagation(state.next) {
return Err(self.convert_no_solution_err(
err,
state.fork_urls,
&state.fork_indexes,
state.env,
&visited,
&self.locations,
&self.capabilities,
));
let result = state.pubgrub.unit_propagation(state.next);
match result {
Err(err) => {
// If unit propagation failed, there is no solution.
return Err(self.convert_no_solution_err(
err,
state.fork_urls,
&state.fork_indexes,
state.env,
&visited,
&self.locations,
&self.capabilities,
));
}
Ok(conflicts) => {
for (affected, incompatibility) in conflicts {
// Conflict tracking: If there was a conflict, track affected and
// culprit for all root cause incompatibilities
state.record_conflict(affected, None, incompatibility);
}
}
}
// Pre-visit all candidate packages, to allow metadata to be fetched in parallel.
@ -359,7 +373,19 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
)?;
}
// Choose a package.
Self::reprioritize_conflicts(&mut state);
trace!(
"assigned packages: {}",
state
.pubgrub
.partial_solution
.extract_solution()
.filter(|(p, _)| !state.pubgrub.package_store[*p].is_proxy())
.map(|(p, v)| format!("{}=={}", state.pubgrub.package_store[p], v))
.join(", ")
);
// Choose a package .
let Some(highest_priority_pkg) =
state.pubgrub.partial_solution.pick_highest_priority_pkg(
|id, _range| state.priorities.get(&state.pubgrub.package_store[id]),
@ -405,6 +431,17 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
resolutions.push(resolution);
continue 'FORK;
};
trace!(
"Chose package for decision: {}. remaining choices: {}",
state.pubgrub.package_store[highest_priority_pkg],
state
.pubgrub
.partial_solution
.undecided_packages()
.filter(|(p, _)| !state.pubgrub.package_store[**p].is_proxy())
.map(|(p, _)| state.pubgrub.package_store[*p].to_string())
.join(", ")
);
highest_priority_pkg
};
@ -642,6 +679,56 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
)
}
/// Change the priority of often conflicting packages and backtrack.
///
/// To be called after unit propagation.
fn reprioritize_conflicts(state: &mut ForkState) {
for package in state.conflict_tracker.prioritize.drain(..) {
let changed = state
.priorities
.mark_conflict_early(&state.pubgrub.package_store[package]);
if changed {
debug!(
"Package {} has too many conflicts (affected), prioritizing",
&state.pubgrub.package_store[package]
);
} else {
debug!(
"Package {} has too many conflicts (affected), already {:?}",
state.pubgrub.package_store[package],
state.priorities.get(&state.pubgrub.package_store[package])
);
}
}
for package in state.conflict_tracker.deprioritize.drain(..) {
let changed = state
.priorities
.mark_conflict_late(&state.pubgrub.package_store[package]);
if changed {
debug!(
"Package {} has too many conflicts (culprit), deprioritizing and backtracking",
state.pubgrub.package_store[package],
);
let backtrack_level = state.pubgrub.backtrack_package(package);
if let Some(backtrack_level) = backtrack_level {
debug!("Backtracked {backtrack_level} decisions");
} else {
debug!(
"Package {} is not decided, cannot backtrack",
state.pubgrub.package_store[package]
);
}
} else {
debug!(
"Package {} has too many conflicts (culprit), already {:?}",
state.pubgrub.package_store[package],
state.priorities.get(&state.pubgrub.package_store[package])
);
}
}
}
/// When trace level logging is enabled, we dump the final
/// set of resolutions, including markers, to help with
/// debugging. Namely, this tells use precisely the state
@ -2198,6 +2285,7 @@ pub(crate) struct ForkState {
/// The top fork has a narrower Python compatibility range, and thus can find a
/// solution that omits Python 3.8 support.
python_requirement: PythonRequirement,
conflict_tracker: ConflictTracker,
}
impl ForkState {
@ -2217,6 +2305,7 @@ impl ForkState {
added_dependencies: FxHashMap::default(),
env,
python_requirement,
conflict_tracker: ConflictTracker::default(),
}
}
@ -2302,7 +2391,7 @@ impl ForkState {
self.priorities.insert(package, version, &self.fork_urls);
}
self.pubgrub.add_package_version_dependencies(
let conflict = self.pubgrub.add_package_version_dependencies(
self.next,
for_version.clone(),
dependencies.into_iter().map(|dependency| {
@ -2314,9 +2403,75 @@ impl ForkState {
(package, version)
}),
);
// Conflict tracking: If the version was rejected due to its dependencies, record culprit
// and affected.
if let Some(incompatibility) = conflict {
self.record_conflict(for_package, Some(for_version), incompatibility);
}
Ok(())
}
fn record_conflict(
&mut self,
affected: Id<PubGrubPackage>,
version: Option<&Version>,
incompatibility: IncompId<PubGrubPackage, Ranges<Version>, UnavailableReason>,
) {
let mut culprit_is_real = false;
for (incompatible, _term) in self.pubgrub.incompatibility_store[incompatibility].iter() {
if incompatible == affected {
continue;
}
if self.pubgrub.package_store[affected].name()
== self.pubgrub.package_store[incompatible].name()
{
// Don't track conflicts between a marker package and the main package, when the
// marker is "copying" the obligations from the main package through conflicts.
continue;
}
culprit_is_real = true;
let culprit_count = self
.conflict_tracker
.culprit
.entry(incompatible)
.or_default();
*culprit_count += 1;
if *culprit_count == CONFLICT_THRESHOLD {
self.conflict_tracker.deprioritize.push(incompatible);
}
}
// Don't track conflicts between a marker package and the main package, when the
// marker is "copying" the obligations from the main package through conflicts.
if culprit_is_real {
if tracing::enabled!(Level::DEBUG) {
let incompatibility = self.pubgrub.incompatibility_store[incompatibility]
.iter()
.map(|(package, _term)| {
format!("{}", self.pubgrub.package_store[package].clone(),)
})
.join(", ");
if let Some(version) = version {
debug!(
"Recording dependency conflict of {}=={} from incompatibility of ({})",
self.pubgrub.package_store[affected], version, incompatibility
);
} else {
debug!(
"Recording unit propagation conflict of {} from incompatibility of ({})",
self.pubgrub.package_store[affected], incompatibility
);
}
}
let affected_count = self.conflict_tracker.affected.entry(self.next).or_default();
*affected_count += 1;
if *affected_count == CONFLICT_THRESHOLD {
self.conflict_tracker.prioritize.push(self.next);
}
}
}
fn add_unavailable_version(&mut self, version: Version, reason: UnavailableVersion) {
// Incompatible requires-python versions are special in that we track
// them as incompatible dependencies instead of marking the package version
@ -3221,3 +3376,19 @@ fn find_conflicting_extra(conflicting: &Conflicts, reqs: &[Requirement]) -> Opti
}
None
}
#[derive(Debug, Default, Clone)]
struct ConflictTracker {
/// How often a decision on the package was discarded due to another package decided earlier.
affected: FxHashMap<Id<PubGrubPackage>, usize>,
/// Package(s) to be prioritized after the next unit propagation
///
/// Distilled from `affected` for fast checking in the hot loop.
prioritize: Vec<Id<PubGrubPackage>>,
/// How often a package was decided earlier and caused another package to be discarded.
culprit: FxHashMap<Id<PubGrubPackage>, usize>,
/// Package(s) to be de-prioritized after the next unit propagation
///
/// Distilled from `culprit` for fast checking in the hot loop.
deprioritize: Vec<Id<PubGrubPackage>>,
}

View File

@ -14433,6 +14433,7 @@ fn lock_explicit_default_index() -> Result<()> {
DEBUG Adding transitive dependency for project==0.1.0: anyio*
DEBUG Searching for a compatible version of anyio (*)
DEBUG No compatible version found for: anyio
DEBUG Recording unit propagation conflict of anyio from incompatibility of (project)
DEBUG Searching for a compatible version of project @ file://[TEMP_DIR]/ (<0.1.0 | >0.1.0)
DEBUG No compatible version found for: project
× No solution found when resolving dependencies:

View File

@ -2905,18 +2905,20 @@ wheels = [
[[package]]
name = "protobuf"
version = "4.25.4"
version = "3.20.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e8/ab/cb61a4b87b2e7e6c312dce33602bd5884797fd054e0e53205f1c27cf0f66/protobuf-4.25.4.tar.gz", hash = "sha256:0dc4a62cc4052a036ee2204d26fe4d835c62827c855c8a03f29fe6da146b380d", size = 380283 }
sdist = { url = "https://files.pythonhosted.org/packages/55/5b/e3d951e34f8356e5feecacd12a8e3b258a1da6d9a03ad1770f28925f29bc/protobuf-3.20.3.tar.gz", hash = "sha256:2e3427429c9cffebf259491be0af70189607f365c2f41c7c3764af6f337105f2", size = 216768 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c8/43/27b48d9040763b78177d3083e16c70dba6e3c3ee2af64b659f6332c2b06e/protobuf-4.25.4-cp310-abi3-win32.whl", hash = "sha256:db9fd45183e1a67722cafa5c1da3e85c6492a5383f127c86c4c4aa4845867dc4", size = 392409 },
{ url = "https://files.pythonhosted.org/packages/0c/d4/589d673ada9c4c62d5f155218d7ff7ac796efb9c6af95b0bd29d438ae16e/protobuf-4.25.4-cp310-abi3-win_amd64.whl", hash = "sha256:ba3d8504116a921af46499471c63a85260c1a5fc23333154a427a310e015d26d", size = 413398 },
{ url = "https://files.pythonhosted.org/packages/34/ca/bf85ffe3dd16f1f2aaa6c006da8118800209af3da160ae4d4f47500eabd9/protobuf-4.25.4-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:eecd41bfc0e4b1bd3fa7909ed93dd14dd5567b98c941d6c1ad08fdcab3d6884b", size = 394160 },
{ url = "https://files.pythonhosted.org/packages/68/1d/e8961af9a8e534d66672318d6b70ea8e3391a6b13e16a29b039e4a99c214/protobuf-4.25.4-cp37-abi3-manylinux2014_aarch64.whl", hash = "sha256:4c8a70fdcb995dcf6c8966cfa3a29101916f7225e9afe3ced4395359955d3835", size = 293700 },
{ url = "https://files.pythonhosted.org/packages/ca/6c/cc7ab2fb3a4a7f07f211d8a7bbb76bba633eb09b148296dbd4281e217f95/protobuf-4.25.4-cp37-abi3-manylinux2014_x86_64.whl", hash = "sha256:3319e073562e2515c6ddc643eb92ce20809f5d8f10fead3332f71c63be6a7040", size = 294612 },
{ url = "https://files.pythonhosted.org/packages/a4/b5/f7e2460dec8347d67e6108bef6ad3291c76e38c898a1087e2c836c02951e/protobuf-4.25.4-cp39-cp39-win32.whl", hash = "sha256:90bf6fd378494eb698805bbbe7afe6c5d12c8e17fca817a646cd6a1818c696ca", size = 392490 },
{ url = "https://files.pythonhosted.org/packages/c7/0b/15bd1a224e5e5744a0dcccf11bcd5dc1405877be38e477b1359d7c2c3737/protobuf-4.25.4-cp39-cp39-win_amd64.whl", hash = "sha256:ac79a48d6b99dfed2729ccccee547b34a1d3d63289c71cef056653a846a2240f", size = 413357 },
{ url = "https://files.pythonhosted.org/packages/b5/95/0ba7f66934a0a798006f06fc3d74816da2b7a2bcfd9b98c53d26f684c89e/protobuf-4.25.4-py3-none-any.whl", hash = "sha256:bfbebc1c8e4793cfd58589acfb8a1026be0003e852b9da7db5a4285bde996978", size = 156464 },
{ url = "https://files.pythonhosted.org/packages/28/55/b80e8567ec327c060fa39b242392e25690c8899c489ecd7bb65b46b7bb55/protobuf-3.20.3-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:f4bd856d702e5b0d96a00ec6b307b0f51c1982c2bf9c0052cf9019e9a544ba99", size = 918427 },
{ url = "https://files.pythonhosted.org/packages/31/be/80a9c6f16dfa4d41be3edbe655349778ae30882407fa8275eb46b4d34854/protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:9aae4406ea63d825636cc11ffb34ad3379335803216ee3a856787bcf5ccc751e", size = 1051042 },
{ url = "https://files.pythonhosted.org/packages/db/96/948d3fcc1fa816e7ae1d27af59b9d8c5c5e582f3994fd14394f31da95b99/protobuf-3.20.3-cp310-cp310-win32.whl", hash = "sha256:28545383d61f55b57cf4df63eebd9827754fd2dc25f80c5253f9184235db242c", size = 780167 },
{ url = "https://files.pythonhosted.org/packages/6f/5e/fc6feb366b0a9f28e0a2de3b062667c521cd9517d4ff55077b8f351ba2f3/protobuf-3.20.3-cp310-cp310-win_amd64.whl", hash = "sha256:67a3598f0a2dcbc58d02dd1928544e7d88f764b47d4a286202913f0b2801c2e7", size = 904029 },
{ url = "https://files.pythonhosted.org/packages/00/e7/d23c439c55c90ae2e52184363162f7079ca3e7d86205b411d4e9dc266f81/protobuf-3.20.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:398a9e0c3eaceb34ec1aee71894ca3299605fa8e761544934378bbc6c97de23b", size = 982826 },
{ url = "https://files.pythonhosted.org/packages/99/25/5825472ecd911f4ac2ac4e9ab039a48b6d03874e2add92fb633e080bf3eb/protobuf-3.20.3-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:bf01b5720be110540be4286e791db73f84a2b721072a3711efff6c324cdf074b", size = 918423 },
{ url = "https://files.pythonhosted.org/packages/c7/df/ec3ecb8c940b36121c7b77c10acebf3d1c736498aa2f1fe3b6231ee44e76/protobuf-3.20.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:daa564862dd0d39c00f8086f88700fdbe8bc717e993a21e90711acfed02f2402", size = 1019250 },
{ url = "https://files.pythonhosted.org/packages/36/8b/433071fed0058322090a55021bdc8da76d16c7bc9823f5795797803dd6d0/protobuf-3.20.3-cp39-cp39-win32.whl", hash = "sha256:819559cafa1a373b7096a482b504ae8a857c89593cf3a25af743ac9ecbd23480", size = 780270 },
{ url = "https://files.pythonhosted.org/packages/11/a5/e52b731415ad6ef3d841e9e6e337a690249e800cc7c06f0749afab26348c/protobuf-3.20.3-cp39-cp39-win_amd64.whl", hash = "sha256:03038ac1cfbc41aa21f6afcbcd357281d7521b4157926f30ebecc8d4ea59dcb7", size = 904215 },
{ url = "https://files.pythonhosted.org/packages/8d/14/619e24a4c70df2901e1f4dbc50a6291eb63a759172558df326347dce1f0d/protobuf-3.20.3-py2.py3-none-any.whl", hash = "sha256:a7ca6d488aa8ff7f329d4c545b2dbad8ac31464f1d8b1c87ad1346717731e4db", size = 162128 },
]
[[package]]
@ -4538,17 +4540,18 @@ wheels = [
[[package]]
name = "tf2onnx"
version = "1.8.4"
version = "1.16.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "flatbuffers" },
{ name = "numpy" },
{ name = "onnx" },
{ name = "protobuf" },
{ name = "requests" },
{ name = "six" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/db/32/33ce509a79c207a39cf04bfa3ec3353da15d1e6553a6ad912f117cc29130/tf2onnx-1.8.4-py3-none-any.whl", hash = "sha256:1ebabb96c914da76e23222b6107a8b248a024bf259d77f027e6690099512d457", size = 345298 },
{ url = "https://files.pythonhosted.org/packages/3f/48/826db3d02645d84e7ee5d5ce8407f771057d40fe224d9c3e89536674ccef/tf2onnx-1.16.1-py3-none-any.whl", hash = "sha256:90fb5f62575896d47884d27dc313cfebff36b8783e1094335ad00824ce923a8a", size = 455820 },
]
[[package]]

View File

@ -1840,20 +1840,20 @@ wheels = [
[[package]]
name = "mypy"
version = "1.11.1"
version = "1.10.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mypy-extensions" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b6/9c/a4b3bda53823439cf395db8ecdda6229a83f9bf201714a68a15190bb2919/mypy-1.11.1.tar.gz", hash = "sha256:f404a0b069709f18bbdb702eb3dcfe51910602995de00bd39cea3050b5772d08", size = 3078369 }
sdist = { url = "https://files.pythonhosted.org/packages/c7/b9/81e4c6dbb1ec1e72503de3ff2c5fe4b7f224e04613b670f8b9004cd8a4dd/mypy-1.10.1.tar.gz", hash = "sha256:1f8f492d7db9e3593ef42d4f115f04e556130f2819ad33ab84551403e97dd4c0", size = 3022304 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0b/b1/62d8ce619493a5364dda4f410912aa12c27126926e8fb8393edca0664640/mypy-1.11.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:7b6343d338390bb946d449677726edf60102a1c96079b4f002dedff375953fc5", size = 10858723 },
{ url = "https://files.pythonhosted.org/packages/fe/aa/2ad15a318bc6a17b7f23e1641a624603949904f6131e09681f40340fb875/mypy-1.11.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:e4fe9f4e5e521b458d8feb52547f4bade7ef8c93238dfb5bbc790d9ff2d770ca", size = 10038078 },
{ url = "https://files.pythonhosted.org/packages/4d/7f/77feb389d91603f55b3c4e3e16ccf8752bce007ed73ca921e42c9a5dff12/mypy-1.11.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:886c9dbecc87b9516eff294541bf7f3655722bf22bb898ee06985cd7269898de", size = 12420213 },
{ url = "https://files.pythonhosted.org/packages/bc/5b/907b4681f68e7ee2e2e88eed65c514cf6406b8f2f83b243ea79bd4eddb97/mypy-1.11.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:fca4a60e1dd9fd0193ae0067eaeeb962f2d79e0d9f0f66223a0682f26ffcc809", size = 12898278 },
{ url = "https://files.pythonhosted.org/packages/5b/b3/2a83be637825d7432b8e6a51e45d02de4f463b6c7ec7164a45009a7cf477/mypy-1.11.1-cp311-cp311-win_amd64.whl", hash = "sha256:0bd53faf56de9643336aeea1c925012837432b5faf1701ccca7fde70166ccf72", size = 9564438 },
{ url = "https://files.pythonhosted.org/packages/f8/d4/4960d0df55f30a7625d9c3c9414dfd42f779caabae137ef73ffaed0c97b9/mypy-1.11.1-py3-none-any.whl", hash = "sha256:0624bdb940255d2dd24e829d99a13cfeb72e4e9031f9492148f410ed30bcab54", size = 2619257 },
{ url = "https://files.pythonhosted.org/packages/38/cf/0645128c6edf70eb9b9687ad42fcb61ea344a7927ed2b78ce2275282fe87/mypy-1.10.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:bd6f629b67bb43dc0d9211ee98b96d8dabc97b1ad38b9b25f5e4c4d7569a0c6a", size = 10740526 },
{ url = "https://files.pythonhosted.org/packages/19/c9/10842953066265e6063c41a85bbee3b877501947c970ea84a1db5f11d32e/mypy-1.10.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a1bbb3a6f5ff319d2b9d40b4080d46cd639abe3516d5a62c070cf0114a457d84", size = 9898375 },
{ url = "https://files.pythonhosted.org/packages/e4/9e/551e897f67c5d67aa1976bc3b4951f297d1daf07250c421bb045b2613350/mypy-1.10.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b8edd4e9bbbc9d7b79502eb9592cab808585516ae1bcc1446eb9122656c6066f", size = 12602338 },
{ url = "https://files.pythonhosted.org/packages/2b/a4/55e3635253e5fa7051674dd5a67582f08b0ba8823e1fdbf7241ed5b32d4e/mypy-1.10.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6166a88b15f1759f94a46fa474c7b1b05d134b1b61fca627dd7335454cc9aa6b", size = 12680741 },
{ url = "https://files.pythonhosted.org/packages/7a/cc/aa881ad051f99915887db0b5de8facc0e224295be22f92178c8f77fd8359/mypy-1.10.1-cp311-cp311-win_amd64.whl", hash = "sha256:5bb9cd11c01c8606a9d0b83ffa91d0b236a0e91bc4126d9ba9ce62906ada868e", size = 9393661 },
{ url = "https://files.pythonhosted.org/packages/2b/ee/d53a3d4792a09b6cd757978951d6dcf8b10825a8b8522b68e9b5eb53b9a1/mypy-1.10.1-py3-none-any.whl", hash = "sha256:71d8ac0b906354ebda8ef1673e5fde785936ac1f29ff6987c7483cfbd5a4235a", size = 2580108 },
]
[[package]]
@ -1867,14 +1867,14 @@ wheels = [
[[package]]
name = "mypy-zope"
version = "0.1.3"
version = "1.0.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mypy" },
{ name = "zope-interface" },
{ name = "zope-schema" },
]
sdist = { url = "https://files.pythonhosted.org/packages/77/45/ba2c0ac5487a70ba26209bc32773ead684092c40e122c5d3c50f8587fa0f/mypy-zope-0.1.3.tar.gz", hash = "sha256:4247f7ad0202cc82f12ee0b14a3d7b31adfeff1b6d5ca176168853d64c672dd1", size = 23600 }
sdist = { url = "https://files.pythonhosted.org/packages/68/8f/4be355338ff798e7cb6afbc01993bd952e834b3718e47eba626d4822b331/mypy_zope-1.0.5.tar.gz", hash = "sha256:2440406d49c0e1199c1cd819c92a2c4957de65579c6abc8a081c927f4bdc8d49", size = 33936 }
[[package]]
name = "myst-parser"

View File

@ -63,6 +63,9 @@ involved packages.
For more details on the PubGrub algorithm, see [Internals of the PubGrub
algorithm](https://pubgrub-rs-guide.pages.dev/internals/intro).
In addition to PubGrub's base algorithm, we also use a heuristic that backtracks and switches the
order of two packages if they have been conflicting too much.
## Forking
Python resolvers historically didn't support backtracking, and even with backtracking, resolution
@ -70,7 +73,7 @@ was usually limited to single environment, which one specific architecture, oper
version, and Python implementation. Some packages use contradictory requirements for different
environments, for example:
```python
```
numpy>=2,<3 ; python_version >= "3.11"
numpy>=1.16,<2 ; python_version < "3.11"
```
@ -85,7 +88,7 @@ In the above example, the partial solution would be split into two resolutions,
If markers overlap or are missing a part of the marker space, the resolver splits additional times —
there can be many forks per package. For example, given:
```python
```
flask > 1 ; sys_platform == 'darwin'
flask > 2 ; sys_platform == 'win32'
flask