Python
/
ruff
mirror of https://github.com/astral-sh/ruff
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
ruff/scripts/ty_benchmark/README.md

1.1 KiB
Raw Permalink Blame History

Getting started

  1. Install uv
  • Unix: curl -LsSf https://astral.sh/uv/install.sh | sh
  • Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
  1. Build ty: cargo build --bin ty --release
  2. cd into the benchmark directory: cd scripts/ty_benchmark
  3. Install Pyright: npm install
  4. Run benchmarks: uv run benchmark

Requires hyperfine 1.20 or newer.

Known limitations

The tested type checkers implement Python's type system to varying degrees and some projects only successfully pass type checking using a specific type checker.

Updating the benchmark

The benchmark script supports snapshoting the results when running with --snapshot and --accept. The goal of those snapshots is to catch accidental regressions. For example, if a project adds new dependencies that we fail to install. They are not intended as a testing tool. E.g. the snapshot runner doesn't account for platform differences so that you might see differences when running the snapshots on your machine.