Improve the performance of the formatter instability check job (#14471)

We should probably get rid of this entirely and subsume it's
functionality in the normal ecosystem checks? I don't think we're using
the black comparison tests anymore, but maybe someone wants it?

There are a few major parts to this:

1. Making the formatter script idempotent, so it can be run repeatedly
and is robust to changing commits
2. Reducing the overhead of the git operations, minimizing the data
transfer
3. Parallelizing all the git operations by repository

This reduces the setup time from 80s to 16s (locally).

The initial motivation for idempotency was to include the repositories
in the GitHub Actions cache. I'm not sure it's worth it yet — they're
about 1GB and would consume our limited cache space. Regardless, it
improves correctness for local invocations.

The total runtime of the job is reduced from ~4m to ~3m.

I also made some cosmetic changes to the output paths and such.
This commit is contained in:
Zanie Blue
2024-11-20 08:55:10 -06:00
committed by GitHub
parent 942d6eeb9f
commit 3c52d2d1bd
2 changed files with 76 additions and 45 deletions

View File

@@ -563,12 +563,12 @@ jobs:
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Formatter progress"
- name: "Run checks"
run: scripts/formatter_ecosystem_checks.sh
- name: "Github step summary"
run: cat target/progress_projects_stats.txt > $GITHUB_STEP_SUMMARY
run: cat target/formatter-ecosystem/stats.txt > $GITHUB_STEP_SUMMARY
- name: "Remove checkouts from cache"
run: rm -r target/progress_projects
run: rm -r target/formatter-ecosystem
check-ruff-lsp:
name: "test ruff-lsp"