Python
/
ruff
mirror of https://github.com/astral-sh/ruff
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
ruff/scripts/ty_benchmark
History
Micha Reiser 747c39a26a
[ty] Add more and update existing projects in `ty_benchmark` (#21536) 		2025-11-25 08:58:34 +01:00
..
snapshots [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
src/benchmark [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
.gitignore [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
README.md [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
package-lock.json [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
package.json [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
pyproject.toml [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00
uv.lock [ty] Add more and update existing projects in `ty_benchmark` (#21536) 2025-11-25 08:58:34 +01:00

README.md

Getting started

  1. Install uv
  • Unix: curl -LsSf https://astral.sh/uv/install.sh | sh
  • Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
  1. Build ty: cargo build --bin ty --release
  2. cd into the benchmark directory: cd scripts/ty_benchmark
  3. Install Pyright: npm install
  4. Run benchmarks: uv run benchmark

Requires hyperfine 1.20 or newer.

Known limitations

The tested type checkers implement Python's type system to varying degrees and some projects only successfully pass type checking using a specific type checker.

Updating the benchmark

The benchmark script supports snapshoting the results when running with --snapshot and --accept. The goal of those snapshots is to catch accidental regressions. For example, if a project adds new dependencies that we fail to install. They are not intended as a testing tool. E.g. the snapshot runner doesn't account for platform differences so that you might see differences when running the snapshots on your machine.