Commit Graph

203 Commits

Author SHA1 Message Date
Charlie Marsh e15b99b911
Rename commands to `pip-sync` and `pip-compile` (#123)
To free up the rest of the interface.
2023-10-18 21:15:20 +00:00
konsti 8cc4fe0d44
Install source distribution requirements with puffin itself instead of pip (#122)
This is also a lot faster. Unfortunately it copies a lot of code from
the sync cli since the `Printer` is private.

The first commit are some refactorings i made when i thought about how i
could reuse the existing code.
2023-10-18 19:11:17 +00:00
Charlie Marsh 7bc42ca2ce
Use `owo_colors` instead of `colored` (#121)
This is what `miette` uses so seems better to avoid two coloring crates.
2023-10-18 18:57:07 +00:00
Charlie Marsh 2d14c0647e
Add a `puffin remove` command (#120) 2023-10-18 18:50:08 +00:00
Charlie Marsh 1fc03780f9
Use `miette` for `puffin add` diagnostics (#119)
Experiment in using `miette` for better user-facing diagnostics in the
CLI crate:

<img width="710" alt="Screen Shot 2023-10-18 at 2 11 54 PM"
src="https://github.com/astral-sh/puffin/assets/1309177/30299da0-da65-4972-944f-cb8cc5f72a77">

For now, only the `add` command has been migrated, and all the library
crates continue to use `anyhow`.
2023-10-18 14:24:09 -04:00
Charlie Marsh 4c87a1d42c
Add a `puffin add` command (#117)
This needs far better error handling and user-facing feedback, but it
does the basic operation (and includes discovery of the `pyproject.toml`
file, etc.).
2023-10-18 00:51:20 -04:00
Charlie Marsh 339553e228
Mark `--no-cache` as global (#116) 2023-10-17 23:15:36 -04:00
Charlie Marsh 5f5788e866
Surface PubGrub derivation trees (#108)
I think the derivation trees could be stronger but this exposes
PubGrub's proof-like error messages.

Closes #102.
2023-10-16 14:14:36 -04:00
Charlie Marsh 7e8ffeb2df
Use `fs-err` in more crates (#100)
Closes https://github.com/astral-sh/puffin/issues/88.
2023-10-16 13:37:58 +00:00
konsti cb29c89424
Better error reporting (#95)
The main change is to print the whole error chain. We can combine this
with adding `.context` to distinct phases to be able to locate crashes
without having to use a debugger.
2023-10-16 02:15:10 +00:00
Charlie Marsh 471a1d657d
Migrate resolver proof-of-concept to PubGrub (#97)
## Summary

This PR enables the proof-of-concept resolver to backtrack by way of
using the `pubgrub-rs` crate.

Rather than using PubGrub as a _framework_ (implementing the
`DependencyProvider` trait, letting PubGrub call us), I've instead
copied over PubGrub's primary solver hook (which is only ~100 lines or
so) and modified it for our purposes (e.g., made it async).

There's a lot to improve here, but it's a start that will let us
understand PubGrub's appropriateness for this problem space. A few
observations:

- In simple cases, the resolver is slower than our current (naive)
resolver. I think it's just that the pipelining isn't as efficient as in
the naive case, where we can just stream package and version fetches
concurrently without any bottlenecks.
- A lot of the code here relates to bridging PubGrub with our own
abstractions -- so we need a `PubGrubPackage`, a `PubGrubVersion`, etc.
2023-10-15 22:05:44 -04:00
konsti 530edb6e39
Add output file option to compile (#93)
`pip-compile` has the same option. I need this esp. since piping doesn't
work as we write to stdout.
2023-10-12 20:42:06 +02:00
konsti 6a7954cdd0
Add `-p` base python option to venv command (#92)
This is the same option that `virtualenv` offers, except that we only
support absolute paths atm and not e.g. `-p 3.10` (which we need to
eventually).
2023-10-12 20:41:52 +02:00
Charlie Marsh 496cb7b2ef
Migrate to `requirements_txt.rs` (#90)
Remove the parser I wrote in favor of Konsti's which is much more
complete. The only change vs. the version in `poc-monotrail` is that I
changed the tests to use insta rather than manually storing and
comparing against JSON snapshots.

Closes https://github.com/astral-sh/puffin/issues/89.
2023-10-12 17:09:00 +00:00
Charlie Marsh 906a482499
Separate unzip into its own install phase (#87) 2023-10-11 15:18:23 +00:00
Charlie Marsh c1fb698eae
Add a separate dist-info name struct (#85) 2023-10-10 23:21:18 +00:00
Charlie Marsh d0764bdc23
Add `puffin venv` command to create virtual environments (#83)
Closes https://github.com/astral-sh/puffin/issues/58.
2023-10-10 13:46:25 -04:00
Charlie Marsh a0294a510c
Rework `puffin sync` output to summarize (#81)
This also moves away from using `tracing` for user-facing logging,
instead introducing a new `Printer` abstraction.

Closes #66.
2023-10-10 03:29:09 +00:00
Charlie Marsh b90140e1bc
Add support for wheel uninstalls (#77)
Closes #36.
2023-10-09 14:14:33 -04:00
Charlie Marsh 485b1dceb6
Use a single requirements iterator in `sync` (#71) 2023-10-09 03:29:38 +00:00
Charlie Marsh ba72950546
Avoid passing cached wheels to the resolver step (#70)
When we go to install a locked `requirements.txt`, if a wheel is already
available in the local cache, and matches the version specifiers, we can
just use it directly without fetching the package metadata. This speeds
up the no-op case by about 33%.

Closes https://github.com/astral-sh/puffin/issues/48.
2023-10-08 22:17:19 -04:00
Charlie Marsh a53f697f62
Use `tracing` for user-facing output (#63)
The setup is now as follows:

- All user-facing logging goes through `tracing` at an `info` leve.
(This excludes messages that go to `stdout`, like the compiled
`requirements.txt` file.)
- We have `--quiet` and `--verbose` command-line flags to set the
tracing filter and format defaults. So if you use `--verbose`, we
include timestamps and targets, and filter at `puffin=debug` level.
- However, we always respect `RUST_LOG`. So you can override the
_filter_ via `RUST_LOG`.

For example: the standard setup filters to `puffin=info`, and doesn't
show timestamps or targets:

<img width="1235" alt="Screen Shot 2023-10-08 at 3 41 22 PM"
src="https://github.com/astral-sh/puffin/assets/1309177/54ca4db6-c66a-439e-bfa3-b86dee136e45">

If you run with `--verbose`, you get debug logging, but confined to our
crates:

<img width="1235" alt="Screen Shot 2023-10-08 at 3 41 57 PM"
src="https://github.com/astral-sh/puffin/assets/1309177/c5c1af11-7f7a-4038-a173-d9eca4c3630b">

If you want verbose logging with _all_ crates, you can add
`RUST_LOG=debug`:

<img width="1235" alt="Screen Shot 2023-10-08 at 3 42 39 PM"
src="https://github.com/astral-sh/puffin/assets/1309177/0b5191f4-4db0-4db9-86ba-6f9fa521bcb6">

I think this is a reasonable setup, though we can see how it feels and
refine over time.

Closes https://github.com/astral-sh/puffin/issues/57.
2023-10-08 15:46:06 -04:00
Charlie Marsh d1ed41170b
Cache environment marker lookups (#55)
Closes https://github.com/astral-sh/puffin/issues/53.
2023-10-08 05:31:19 +00:00
Charlie Marsh fd5aef2c75
Avoid error when repeatedly clearing cache (#51)
Also avoid failing to clear the cache when it contains non-directories
(e.g., I had a `.DS_Store` after looking at it in Finder).
2023-10-08 04:16:48 +00:00
Charlie Marsh 92160e37df
Surface error when unable to find package (#45) 2023-10-07 19:43:12 +00:00
Charlie Marsh 9be02d1590
Skip already-installed dependencies during `sync` command (#43)
Closes https://github.com/astral-sh/puffin/issues/35.
2023-10-07 19:26:45 +00:00
Charlie Marsh bc1736feff
Add a `freeze` command to list installed dependencies (#42)
A pre-requisite for https://github.com/astral-sh/puffin/issues/35.
2023-10-07 18:46:09 +00:00
Charlie Marsh f3015ffc1f
Add a `clean` command to clear the cache (#41) 2023-10-07 15:19:03 +00:00
Charlie Marsh 162952bf64
Add a content-addressed cache for wheels (#38)
Closes https://github.com/astral-sh/puffin/issues/4.
2023-10-07 14:24:52 +00:00
Charlie Marsh ae28552b3a
Use local copy of `install-wheel-rs` (#34)
This PR modifies the `install-wheel-rs` (and a few other crates) to get
everything playing nicely. Specifically, CI should pass, and all these
crates now use workspace dependencies between one another.

As part of this change, I split out the wheel name parsing into its own
`wheel-filename` crate, and the compatibility tag parsing into its own
`platform-tags` crate.
2023-10-07 01:43:55 +00:00
Charlie Marsh dd26cfa0cc
Migrate to `tokio` (#27)
Closes https://github.com/astral-sh/puffin/issues/26.
2023-10-06 20:31:03 +00:00
Charlie Marsh dab70a661a
Change `install` to `sync` (with sync semantics) (#24)
For better separate at this stage (and following `pip-tools`), it's now
`puffin sync`, and it assumes `--no-deps`.
2023-10-06 19:42:58 +00:00
Charlie Marsh ff8e24a621
Move `puffin-installer` to its own crate (#23) 2023-10-06 19:31:21 +00:00
Charlie Marsh f395c9c98c Update README 2023-10-06 01:03:07 -04:00
Charlie Marsh 28721cf5fc Avoid caching wheel fetches 2023-10-06 00:50:30 -04:00
Charlie Marsh a43328d914
Support wheel installation (#19)
Closes https://github.com/astral-sh/puffin/issues/8.
2023-10-06 00:47:45 -04:00
Charlie Marsh 47bbb7a78e
Separate platform tags (#18) 2023-10-05 23:24:38 -04:00
Charlie Marsh 9ea6eaeb10
Add separate compile and install commands (#17)
Closes #9.
2023-10-05 21:44:31 -04:00
Charlie Marsh 8b151a64d5
Rename `puffin-requirements` to `puffin-package` (#16)
Closes https://github.com/astral-sh/puffin/issues/7.
2023-10-05 21:03:20 -04:00
Charlie Marsh 94895de46d
Add support for wheel tag parsing (#15)
Closes https://github.com/astral-sh/puffin/issues/12.
2023-10-05 20:59:58 -04:00
Charlie Marsh 2d6266b167
Add an HTTP cache (and `--no-cache` argument) (#14)
Closes https://github.com/astral-sh/puffin/issues/3.
2023-10-05 19:14:05 -04:00
Charlie Marsh 1063d8c150
Add Python interpreter detection (#11)
Closes https://github.com/astral-sh/puffin/issues/2.
2023-10-05 15:09:22 -04:00
Charlie Marsh b059c590c4
Add basic CI via GitHub Actions (#10)
Closes https://github.com/astral-sh/puffin/issues/1.
2023-10-05 13:42:58 -04:00
Charlie Marsh b4828fb3f2 Remove progress bar 2023-10-05 12:45:38 -04:00
Charlie Marsh 7f497fa43f Add progress bar 2023-10-05 12:45:38 -04:00
Charlie Marsh 8032d4606e Misc. changes 2023-10-05 12:45:38 -04:00
Charlie Marsh f51432382a Do basic resolution 2023-10-05 12:45:38 -04:00
Charlie Marsh 0f10595ac3 Add version selection 2023-10-05 12:45:38 -04:00
Charlie Marsh 44b444494e Fetch package metadata in parallel 2023-10-05 12:45:38 -04:00
Charlie Marsh b08e8c78b5 Remove normalized representation of SimpleJson 2023-10-05 12:45:38 -04:00
Charlie Marsh 1a2f35801b Add client networking stack 2023-10-05 12:45:38 -04:00
Charlie Marsh 53607df7c6 Add a requirements.txt parser 2023-10-05 12:45:38 -04:00
Charlie Marsh 8b9ac30507 Add license, Cargo.toml, etc. 2023-10-05 12:45:38 -04:00