Files
uv/.github/workflows/test.yml
Zanie Blue f9ee1c2bf4 Add nextest partitioning to Windows tests (#17421)
Yet another attempt at #5714 to see if it improves CI times now that
Windows test times have increased

I think this is worth it now

```
main   (1 shard):  13m 46s
branch (2 shards): 10m 31s
branch (3 shards):  7m 34s
```

total time (a.k.a., cost increase)

```
main   (1 shard):  13m 46s
branch (2 shards): 20m 18s
branch (3 shards): 21m 32s
```

As in #875, we could explore moving the build step before the sharded
test runs. The build is 3m of the runtime, but I think artifact transfer
and runner startup time was too expensive last time and I don't expect
it to be faster. It might save a bit on CI cost, but I'm not super
worried about that.

I chose three shards due to a reasonable reduction in per-shard runtime
without a big increase in total runtime (compared to two shards). I
think since half of the shard runtime is fixed build time (vs reducable
test time), going up to more shards would be wasteful.

We use a hash-based strategy for test splitting, which means adding
tests will never move tests across shards.

I moved _off_ of the Depot runner because the "checkout" setup is taking
_way_ too long, about 120s more than the GitHub one which, once we split
over multiple shards, overcomes the speed benefits of the Depot runners.

The downsides here are:

1. Increased total CI time and therefore increased cost
2. GitHub runners are more expensive than Depot runners
3. Failing tests can be split across multiple shards, requiring more
GitHub UI navigation

(1) The runtime is unacceptably slow and I think the cost increase is
fairly marginal
(2) We can move back to Depot runners once they resolve the specific
performance issue
(3) Failing tests on Windows without failing tests on Linux are fairly
rare and should often be isolated to a single shard

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-13 09:20:06 -06:00

171 lines
5.6 KiB
YAML

on:
workflow_call:
permissions: {}
env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always
PYTHON_VERSION: "3.12"
RUSTUP_MAX_RETRIES: 10
RUST_BACKTRACE: 1
jobs:
# We use the large GitHub actions runners
# For Ubuntu and Windows, this requires Organization-level configuration
# See: https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#about-ubuntu-and-windows-larger-runners
cargo-test-linux:
timeout-minutes: 10
runs-on: depot-ubuntu-22.04-16
name: "cargo test on linux"
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: "Install Rust toolchain"
run: rustup show
- uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
with:
version: "0.9.24"
- name: "Install required Python versions"
run: uv python install
- name: "Install secret service"
run: |
sudo apt update -y
sudo apt install -y gnome-keyring
- name: "Start gnome-keyring"
# run gnome-keyring with 'foobar' as password for the login keyring
# this will create a new login keyring and unlock it
# the login password doesn't matter, but the keyring must be unlocked for the tests to work
run: gnome-keyring-daemon --components=secrets --daemonize --unlock <<< 'foobar'
- name: "Install cargo nextest"
uses: taiki-e/install-action@d850aa816998e5cf15f67a78c7b933f2a5033f8a # v2.63.3
with:
tool: cargo-nextest
- name: "Cargo test"
env:
# Retry more than default to reduce flakes in CI
UV_HTTP_RETRIES: 5
run: |
cargo nextest run \
--cargo-profile fast-build \
--features python-patch,native-auth,secret-service \
--workspace \
--profile ci-linux
cargo-test-macos:
timeout-minutes: 20
# Only run macOS tests on main without opt-in
if: ${{ contains(github.event.pull_request.labels.*.name, 'test:macos') || github.ref == 'refs/heads/main' }}
runs-on: depot-macos-14
name: "cargo test on macos"
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: "Install Rust toolchain"
run: rustup show
- uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
with:
version: "0.9.24"
- name: "Install required Python versions"
run: uv python install
- name: "Install cargo nextest"
uses: taiki-e/install-action@d850aa816998e5cf15f67a78c7b933f2a5033f8a # v2.63.3
with:
tool: cargo-nextest
- name: "Cargo test"
env:
# Retry more than default to reduce flakes in CI
UV_HTTP_RETRIES: 5
run: |
cargo nextest run \
--cargo-profile fast-build \
--no-default-features \
--features python,python-managed,pypi,git,git-lfs,performance,crates-io,native-auth,apple-native \
--workspace \
--profile ci-macos
cargo-test-windows:
timeout-minutes: 15
runs-on: github-windows-2025-x86_64-16
name: "cargo test on windows ${{ matrix.partition }} of 3"
strategy:
fail-fast: false
matrix:
partition: [1, 2, 3]
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Setup Dev Drive
run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1
# actions/checkout does not let us clone into anywhere outside ${{ github.workspace }}, so we have to copy the clone...
- name: Copy Git Repo to Dev Drive
run: |
Copy-Item -Path "${{ github.workspace }}" -Destination "$Env:UV_WORKSPACE" -Recurse
- uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
with:
version: "0.9.24"
- name: "Install required Python versions"
run: uv python install
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
with:
workspaces: ${{ env.UV_WORKSPACE }}
# Use the same cache entry across the partitions
add-job-id-key: false
- name: "Install Rust toolchain"
working-directory: ${{ env.UV_WORKSPACE }}
run: rustup show
- name: "Install cargo nextest"
uses: taiki-e/install-action@d850aa816998e5cf15f67a78c7b933f2a5033f8a # v2.63.3
with:
tool: cargo-nextest
- name: "Cargo test"
working-directory: ${{ env.UV_WORKSPACE }}
env:
# Retry more than default to reduce flakes in CI
UV_HTTP_RETRIES: 5
# Avoid permission errors during concurrent tests
# See https://github.com/astral-sh/uv/issues/6940
UV_LINK_MODE: copy
shell: bash
run: |
cargo nextest run \
--cargo-profile fast-build \
--no-default-features \
--features python,pypi,python-managed,native-auth,windows-native \
--workspace \
--profile ci-windows \
--partition hash:${{ matrix.partition }}/3