8116f6131a
Profiling is already extensively documented in ruff, so we can just link to that guide. I added `tracing-durations-export` to the guide because i found it a useful tool for optimizing.
3.3 KiB
Contributing
Setup
Rust, a C compiler, and CMake are required to build Puffin.
Testing Puffin requires multiple specific Python versions. We provide a script to bootstrap development by downloading the required versions.
Linux
On Ubuntu and other Debian-based distributions, you can install the C compiler and CMake with
sudo apt install build-essential cmake
macOS
CMake may be installed with Homebrew:
brew install cmake
The Python bootstrapping script requires coreutils and zstd; we recommend installing them with Homebrew:
brew install coreutils zstd
See the Python section for instructions on installing the Python versions.
Windows
You can install CMake from the installers or with pipx install cmake (make sure that the pipx install path is in PATH, pipx complains if it isn't).
Python
Install required Python versions with the bootstrapping script:
scripts/bootstrap/install.sh
The installed Python binaries will be available in <repo>/bin and must be added to your path to be used. We
provide a .env file with the proper environment variables for development. You may activate it with:
source .env
Or, if you use direnv to manage your environment:
echo "dotenv" >> .envrc
direnv allow
Testing
To run the tests we recommend nextest. Make sure to run the tests with --all-features, otherwise you'll miss most of our integration tests.
Running inside a docker container
Source distributions can run arbitrary code on build and can make unwanted modifications to your system (https://moyix.blogspot.com/2022/09/someones-been-messing-with-my-subnormals.html, https://pypi.org/project/nvidia-pyindex/), which can even occur when just resolving requirements. To prevent this, there's a Docker container you can run commands in:
docker buildx build -t puffin-builder -f builder.dockerfile --load .
# Build for musl to avoid glibc errors, might not be required with your OS version
cargo build --target x86_64-unknown-linux-musl --profile profiling --features vendored-openssl
docker run --rm -it -v $(pwd):/app puffin-builder /app/target/x86_64-unknown-linux-musl/profiling/puffin-dev resolve-many --cache-dir /app/cache-docker /app/scripts/popular_packages/pypi_10k_most_dependents.txt
We recommend using this container if you don't trust the dependency tree of the package(s) you are trying to resolve or install.
Profiling
Please refer to Ruff's Profiling Guide, it applies to Puffin, too.
Analysing concurrency
You can use tracing-durations-export to visualize parallel requests and find any spots where Puffin is CPU-bound. Example usage, with puffin and puffin-dev respectively:
RUST_LOG=puffin=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --profile profiling -- pip compile scripts/requirements/jupyter.in
RUST_LOG=puffin=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --bin puffin-dev --profile profiling -- resolve jupyter