1dc9904f8c
Run `cargo test` on windows in CI, pulling the switch on tier 1 windows support. These changes make the bootstrap script virtually required for running the tests. This gives us consistency between and CI, but it also locks our tests to python-build-standalone and an articificial `PATH`. I've deleted the shell bootstrap script in favor of only the python one, which also runs on windows. I've left the (sym)link creation of the bootstrap in place, even though it is not used by the tests anymore. I've reactivated the three tests that would previously stack overflow by doubling their stack sizes. The stack overflows only happen in debug mode, so this is neither a user facing problem nor an actual problem with our code and this workaround seems better than optimizing our code for case that the (release) compiler can optimize much better for. The handling of patch versions will be fixed in a follow-up PR. Closes #1160 Closes #1161 --------- Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2.9 KiB
Contributing
Setup
Rust, a C compiler, and CMake are required to build Puffin.
Linux
On Ubuntu and other Debian-based distributions, you can install the C compiler and CMake with
sudo apt install build-essential cmake
macOS
CMake may be installed with Homebrew:
brew install cmake
The Python bootstrapping script requires coreutils and zstd; we recommend installing them with Homebrew:
brew install coreutils zstd
See the Python section for instructions on installing the Python versions.
Windows
You can install CMake from the installers or with pipx install cmake (make sure that the pipx install path is in PATH, pipx complains if it isn't).
Testing
Testing Puffin requires multiple specific Python versions. You can install them into
<project root>/bin via our bootstrapping script:
pipx run scripts/bootstrap/install.py
Alternatively, you can install zstandard from PyPI, then run:
python3.12 scripts/bootstrap/install.py
For running tests, we recommend nextest.
Running inside a docker container
Source distributions can run arbitrary code on build and can make unwanted modifications to your system (https://moyix.blogspot.com/2022/09/someones-been-messing-with-my-subnormals.html, https://pypi.org/project/nvidia-pyindex/), which can even occur when just resolving requirements. To prevent this, there's a Docker container you can run commands in:
docker buildx build -t puffin-builder -f builder.dockerfile --load .
# Build for musl to avoid glibc errors, might not be required with your OS version
cargo build --target x86_64-unknown-linux-musl --profile profiling --features vendored-openssl
docker run --rm -it -v $(pwd):/app puffin-builder /app/target/x86_64-unknown-linux-musl/profiling/puffin-dev resolve-many --cache-dir /app/cache-docker /app/scripts/popular_packages/pypi_10k_most_dependents.txt
We recommend using this container if you don't trust the dependency tree of the package(s) you are trying to resolve or install.
Profiling
Please refer to Ruff's Profiling Guide, it applies to Puffin, too.
Analysing concurrency
You can use tracing-durations-export to visualize parallel requests and find any spots where Puffin is CPU-bound. Example usage, with puffin and puffin-dev respectively:
RUST_LOG=puffin=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --profile profiling -- pip compile scripts/requirements/jupyter.in
RUST_LOG=puffin=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --bin puffin-dev --profile profiling -- resolve jupyter