## Summary
This PR adds a playground for Red Knot
[Screencast from 2024-08-14
10-33-54.webm](https://github.com/user-attachments/assets/ae81d85f-74a3-4ba6-bb61-4a871b622f05)
Sharing does work 😆 I just forgot to start wrangler.
It supports:
* Multiple files
* Showing the AST
* Showing the tokens
* Sharing
* Persistence to local storage
Future extensions:
* Configuration support: The `pyproject.toml` would *just* be another
file.
* Showing type information on hover
## Blockers
~~Salsa uses `catch_unwind` to break cycles, which Red Knot uses
extensively when inferring types in the standard library.
However, WASM (at least `wasm32-unknown-unknown`) doesn't support
`catch_unwind` today, so the playground always crashes when the type
inference encounters a cycle.~~
~~I created a discussion in the [salsa
zulip](https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/WASM.20support)
to see if it would be possible to **not** use catch unwind to break
cycles.~~
~~[Rust tracking issue for WASM catch unwind
support](https://github.com/rust-lang/rust/issues/118168)~~
~~I tried to build the WASM with the nightly compiler option but ran
into problems because wasm-bindgen doesn't support WASM-exceptions. We
could try to write the binding code by hand.~~
~~Another alternative is to use `wasm32-unknown-emscripten` but it's
rather painful to build~~
## Summary
This is prep-work for the Red Knot playground. We'll have two
playgrounds, one for Red Knot and Ruff.
I want to share some components between the two, a "shared" NPM package
in a local workspace is a great fit for that.
I also want to share the dev dependencies and dev scripts. Again, NPM
workspaces are great for that.
This PR also sets up a CI workflow for the playground to prevent
surprises during the release.
## Test Plan
CI, local `npm install`, `npm start`, ...
I verified that the new CI step fails if there's a typescript or
formatting error.
* [Deployment test
run](https://github.com/astral-sh/ruff/actions/runs/13904914480/job/38905524353)
## Summary
The PPC64le wheel testing job spuriously failes due to some race when
installing python dependencies.
This is very annoying because it requires restarting the release process
over and over again until you're lucky and it passes.
This PR disables wheel testing on PPC64le
This is the same as we did in uv, see
https://github.com/astral-sh/uv/issues/11231
## Test Plan
The wheel test step was skipped in CI, see
https://github.com/astral-sh/ruff/actions/runs/13895143309/job/38874065160?pr=16793
but it still runs for other targets
## Summary
Use bash and `git diff` to determine which steps need to run.
We previously used the `changed-files` github actions but using `git`
directly seems simple enough.
All credit for the bash magic goes to @zanieb and @geofft. All I did was
replace the paths arguments.
## Test Plan
* [Linter only change](https://github.com/astral-sh/ruff/pull/16800):
See how the fuzzer and formatter steps, and the linter ecosystem checks
are skipped
* [Formatter only change](https://github.com/astral-sh/ruff/pull/16799):
See how the fuzzer and linter ecosystem checks are skipped
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[cloudflare/wrangler-action](https://redirect.github.com/cloudflare/wrangler-action)
| action | patch | `v3.14.0` -> `v3.14.1` |
---
> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.
---
### Release Notes
<details>
<summary>cloudflare/wrangler-action
(cloudflare/wrangler-action)</summary>
###
[`v3.14.1`](https://redirect.github.com/cloudflare/wrangler-action/releases/tag/v3.14.1)
[Compare
Source](https://redirect.github.com/cloudflare/wrangler-action/compare/v3.14.0...v3.14.1)
##### Patch Changes
-
[#​358](https://redirect.github.com/cloudflare/wrangler-action/pull/358)
[`cd6314a`](cd6314a97b)
Thanks [@​penalosa](https://redirect.github.com/penalosa)! - Use
`secret bulk` instead of deprecated `secret:bulk` command
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [x] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yMDAuMCIsInVwZGF0ZWRJblZlciI6IjM5LjIwMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
## Summary
The intent here is that all actions should be pinned to an immutable SHA
(but that Renovate should annotate each SHA with the corresponding
SemVer version).
See https://github.com/astral-sh/uv/pull/12189
## Test plan
```
npx --yes --package renovate -- renovate-config-validator
npm warn deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated rimraf@2.4.5: Rimraf versions prior to v4 are no longer supported
npm warn deprecated boolean@3.2.0: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
npm warn deprecated glob@6.0.4: Glob versions prior to v9 are no longer supported
INFO: Validating .github/renovate.json5
INFO: Config validated successfully
```
## Summary
tj-actions/changed-files no longer exists due to a malicious commit.
This PR removes it so that we can re-enable CI.
We can follow up with a proper replacement in a separate PR
## Summary
The ecosystem checks have proven useful so far, so I'm extending the
list a bit. My main selection criteria are:
- Few dependencies (we don't understand -stubs/-types packages yet)
- Fewer than 1000 diagnostics
- No panics
## Test Plan
Ran it locally. We now have ~2k diagnostics in total, across 12 projects
## Summary
I noticed that the pipeline can succeed if there are problems with tool
installation or dependency resolution. This change makes sure that the
pipeline fails in these cases.
## Summary
Add a new pipeline to comment on PRs if there is a mypy_primer diff
result.
## Test Plan
Not yet, I'm afraid I will have to merge this first to have the pipeline
available on main.
Regardless of whether #16408 and #16311 pan out, this part is worth
pulling out as a separate PR.
Before, you had to define a new `IndexVec` index type for each type of
association list you wanted to create. Now there's a single index type
that's internal to the alist implementation, and you use `List<K, V>` to
store a handle to a particular list.
This also adds some property tests for the alist implementation.
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[cloudflare/wrangler-action](https://redirect.github.com/cloudflare/wrangler-action)
| action | minor | `v3.13.1` -> `v3.14.0` |
---
> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.
---
### Release Notes
<details>
<summary>cloudflare/wrangler-action
(cloudflare/wrangler-action)</summary>
###
[`v3.14.0`](https://redirect.github.com/cloudflare/wrangler-action/releases/tag/v3.14.0)
[Compare
Source](https://redirect.github.com/cloudflare/wrangler-action/compare/v3.13.1...v3.14.0)
##### Minor Changes
-
[#​351](https://redirect.github.com/cloudflare/wrangler-action/pull/351)
[`4ff07f4`](4ff07f4310)
Thanks [@​Maximo-Guk](https://redirect.github.com/Maximo-Guk)! -
Use wrangler outputs for version upload and wrangler deploy
##### Patch Changes
-
[#​350](https://redirect.github.com/cloudflare/wrangler-action/pull/350)
[`e209094`](e209094e62)
Thanks [@​Maximo-Guk](https://redirect.github.com/Maximo-Guk)! -
Handle failures in createGitHubDeployment and createGitHubJobSummary
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNjcuMSIsInVwZGF0ZWRJblZlciI6IjM5LjE2Ny4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR replaces most of the hard-coded AST definitions with a
generation script, similar to what happens in `rust_python_formatter`.
I've replaced every "rote" definition that I could find, where the
content is entirely boilerplate and only depends on what syntax nodes
there are and which groups they belong to.
This is a pretty massive diff, but it's entirely a refactoring. It
should make absolutely no changes to the API or implementation. In
particular, this required adding some configuration knobs that let us
override default auto-generated names where they don't line up with
types that we created previously by hand.
## Test plan
There should be no changes outside of the `rust_python_ast` crate, which
verifies that there were no API changes as a result of the
auto-generation. Aggressive `cargo clippy` and `uvx pre-commit` runs
after each commit in the branch.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
The symlink-approach in the typeshed-sync workflow caused some problems
on Windows, even though it seemed to work fine in CI:
https://github.com/astral-sh/ruff/pull/15138#issuecomment-2578642129
Here, we rely on `build.rs` to patch typeshed instead, which allows us
to get rid of the modifications in the workflow (thank you
@MichaReiser for the idea).
## Test Plan
- Made sure that changes to `knot_extensions.pyi` result in a recompile
of `red_knot_vendored`.
## Summary
- Add a workflow to run property tests on a daily basis (based on
`daily_fuzz.yaml`)
- Mark `assignable_to_is_reflexive` as flaky (related to #14899)
- Add new (failing) `intersection_assignable_to_both` test (also related
to #14899)
## Test Plan
Ran:
```bash
export QUICKCHECK_TESTS=100000
while cargo test --release -p red_knot_python_semantic -- \
--ignored types::property_tests::stable; do :; done
```
Observed successful property_tests CI run
## Summary
Adds a type-check-time Python API that allows us to create and
manipulate types and to test various of their properties. For example,
this can be used to write a Markdown test to make sure that `A & B` is a
subtype of `A` and `B`, but not of an unrelated class `C` (something
that requires quite a bit more code to do in Rust):
```py
from knot_extensions import Intersection, is_subtype_of, static_assert
class A: ...
class B: ...
type AB = Intersection[A, B]
static_assert(is_subtype_of(AB, A))
static_assert(is_subtype_of(AB, B))
class C: ...
static_assert(not is_subtype_of(AB, C))
```
I think this functionality is also helpful for interactive debugging
sessions, in order to query various properties of Red Knot's type
system. Which is something that otherwise requires a custom Rust unit
test, some boilerplate code and constant re-compilation.
## Test Plan
- New Markdown tests
- Tested the modified typeshed_sync workflow locally
## Summary
This PR upgrades zizmor to the latest release in our CI. zizmor is a
static analyzer checking for security issues in GitHub workflows. The
new release finds some new issues in our workflows; this PR fixes some
of the issues, and adds ignores for some other issues.
The issues fixed in this PR are new cases of zizmor's
[`template-injection`](https://woodruffw.github.io/zizmor/audits/#template-injection)
rule being emitted. The issues I'm ignoring for now are all to do with
the
[`cache-poisoning`](https://woodruffw.github.io/zizmor/audits/#cache-poisoning)
rule. The main reason I'm fixing some but ignoring others is that I'm
confident fixing the template-injection diagnostics won't have any
impact on how our workflows operate in CI, but I'm worried that fixing
the cache-poisoning diagnostics could slow down our CI a fair bit. I
don't mind if somebody else is motivated to try to fix these
diagnostics, but for now I think I'd prefer to just ignore them; it
doesn't seem high-priority enough to try to fix them right now :-)
## Test Plan
- `uvx pre-commit run -a --hook-stage=manual` passes locally
- Let's see if CI passes on this PR...
## Summary
This PR changes our zizmor configuration to also flag low-severity
security issues in our GitHub Actions workflows. It's a followup to
https://github.com/astral-sh/ruff/pull/14844. The issues being fixed
here were all flagged by [zizmor's `template-injection`
rule](https://woodruffw.github.io/zizmor/audits/#template-injection):
> Detects potential sources of code injection via template expansion.
>
> GitHub Actions allows workflows to define template expansions, which
occur within special `${{ ... }}` delimiters. These expansions happen
before workflow and job execution, meaning the expansion of a given
expression appears verbatim in whatever context it was performed in.
>
> Template expansions aren't syntax-aware, meaning that they can result
in unintended shell injection vectors. This is especially true when
they're used with attacker-controllable expression contexts, such as
`github.event.issue.title` (which the attacker can fully control by
supplying a new issue title).
[...]
> To fully remediate the vulnerability, you should not use `${{
env.VARNAME }}`, since that is still a template expansion. Instead, you
should use `${VARNAME}` to ensure that the shell itself performs the
variable expansion.
## Test Plan
I tested that this passes all zizmore warnings by running `pre-commit
run -a zizmor` locally. The other test is obviously to check that the
workflows all still run correctly in CI 😄
## Summary
A [recent exploit](https://github.com/advisories/GHSA-7x29-qqmq-v6qc)
brought attention to how easy it can be for attackers to use template
expansion in GitHub Actions workflows to inject arbitrary code into a
repository. That vulnerability [would have been caught by the zizmor
linter](https://blog.yossarian.net/2024/12/06/zizmor-ultralytics-injection),
which looks for potential security vulnerabilities in GitHub Actions
workflows. This PR adds [zizmor](https://github.com/woodruffw/zizmor) as
a pre-commit hook and fixes the high- and medium-severity warnings
flagged by the tool.
All the warnings fixed in this PR are related to this zizmor check:
https://woodruffw.github.io/zizmor/audits/#artipacked. The summary of
the check is that `actions/checkout` will by default persist git
configuration for the duration of the workflow, which can be insecure.
It's unnecessary unless you actually need to do things with `git` later
on in the workflow. None of our workflows do except for
`publish-docs.yml` and `sync-typeshed.yml`, so I set
`persist-credentials: true` for those two but `persist-credentials:
false` for all other uses of `actions/checkout`.
Unfortunately there are several warnings in `release.yml`, including
four high-severity warnings. However, this is a generated workflow file,
so I have deliberately excluded this file from the check. These are the
findings in `release.yml`:
<details>
<summary>release.yml findings</summary>
```
warning[artipacked]: credential persistence through GitHub Actions artifacts
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:62:9
|
62 | - uses: actions/checkout@v4
| _________-
63 | | with:
64 | | submodules: recursive
| |_______________________________- does not set persist-credentials: false
|
= note: audit confidence → Low
warning[artipacked]: credential persistence through GitHub Actions artifacts
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:124:9
|
124 | - uses: actions/checkout@v4
| _________-
125 | | with:
126 | | submodules: recursive
| |_______________________________- does not set persist-credentials: false
|
= note: audit confidence → Low
warning[artipacked]: credential persistence through GitHub Actions artifacts
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:174:9
|
174 | - uses: actions/checkout@v4
| _________-
175 | | with:
176 | | submodules: recursive
| |_______________________________- does not set persist-credentials: false
|
= note: audit confidence → Low
warning[artipacked]: credential persistence through GitHub Actions artifacts
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:249:9
|
249 | - uses: actions/checkout@v4
| _________-
250 | | with:
251 | | submodules: recursive
252 | | # Create a GitHub Release while uploading all files to it
| |_______________________________________________________________- does not set persist-credentials: false
|
= note: audit confidence → Low
error[excessive-permissions]: overly broad workflow or job-level permissions
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:17:1
|
17 | / permissions:
18 | | "contents": "write"
... |
39 | | # If there's a prerelease-style suffix to the version, then the release(s)
40 | | # will be marked as a prerelease.
| |_________________________________^ contents: write is overly broad at the workflow level
|
= note: audit confidence → High
error[template-injection]: code injection via template expansion
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:80:9
|
80 | - id: plan
| _________^
81 | | run: |
| |_________^
82 | || dist ${{ (inputs.tag && inputs.tag != 'dry-run' && format('host --steps=create --tag={0}', inputs.tag)) || 'plan' }} --out...
83 | || echo "dist ran successfully"
84 | || cat plan-dist-manifest.json
85 | || echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
| ||__________________________________________________________________________________^ this step
| ||__________________________________________________________________________________^ inputs.tag may expand into attacker-controllable code
|
= note: audit confidence → Low
error[template-injection]: code injection via template expansion
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:80:9
|
80 | - id: plan
| _________^
81 | | run: |
| |_________^
82 | || dist ${{ (inputs.tag && inputs.tag != 'dry-run' && format('host --steps=create --tag={0}', inputs.tag)) || 'plan' }} --out...
83 | || echo "dist ran successfully"
84 | || cat plan-dist-manifest.json
85 | || echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
| ||__________________________________________________________________________________^ this step
| ||__________________________________________________________________________________^ inputs.tag may expand into attacker-controllable code
|
= note: audit confidence → Low
error[template-injection]: code injection via template expansion
--> /Users/alexw/dev/ruff/.github/workflows/release.yml:80:9
|
80 | - id: plan
| _________^
81 | | run: |
| |_________^
82 | || dist ${{ (inputs.tag && inputs.tag != 'dry-run' && format('host --steps=create --tag={0}', inputs.tag)) || 'plan' }} --out...
83 | || echo "dist ran successfully"
84 | || cat plan-dist-manifest.json
85 | || echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
| ||__________________________________________________________________________________^ this step
| ||__________________________________________________________________________________^ inputs.tag may expand into attacker-controllable code
|
= note: audit confidence → Low
```
</details>
## Test Plan
`uvx pre-commit run -a`
## Summary
This PR adds a fuzzer harness for red knot that runs the type checker on
source code that contains invalid syntax.
Additionally, this PR also updates the `init-fuzzer.sh` script to
increase the corpus size to:
* Include various crates that includes Python source code
* Use the 3.13 CPython source code
And, remove any non-Python files from the final corpus so that when the
fuzzer tries to minify the corpus, it doesn't produce files that only
contains documentation content as that's just noise.
## Test Plan
Run `./fuzz/init-fuzzer.sh`, say no to the large dataset.
Run the fuzzer with `cargo +night fuzz run red_knot_check_invalid_syntax
-- -timeout=5`
## Summary
Make sure we run the tests for mdtest-only changes.
## Test Plan
Tested if positive glob patterns override negative patterns here:
https://codepen.io/mrmlnc/pen/OXQjMe
## Summary
This PR gets rid of the `requirements.in` and `requirements.txt` files
in the `scripts/fuzz-parser` directory, and replaces them with
`pyproject.toml` and `uv.lock` files. The script is renamed from
`fuzz-parser` to `py-fuzzer` (since it can now also be used to fuzz
red-knot as well as the parser, following
https://github.com/astral-sh/ruff/pull/14566), and moved from the
`scripts/` directory to the `python/` directory, since it's now a
(uv)-pip-installable project in its own right.
I've been resisting this for a while, because conceptually this script
just doesn't feel "complicated" enough to me for it to be a full-blown
package. However, I think it's time to do this. Making it a proper
package has several advantages:
- It means we can run it from the project root using `uv run` without
having to activate a virtual environment and ensure that all required
dependencies are installed into that environment
- Using a `pyproject.toml` file means that we can express that the
project requires Python 3.12+ to run properly; this wasn't possible
before
- I've been running mypy on the project locally when I've been working
on it or reviewing other people's PRs; now I can put the mypy config for
the project in the `pyproject.toml` file
## Test Plan
I manually tested that all the commands detailed in
`python/py-fuzzer/README.md` work for me locally.
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
## Summary
This is about the easiest patch that I can think of. It has a drawback
in that there is no real guarantee this won't happen again. I think this
might be acceptable, given that all of this is a temporary thing.
And we also add a new CI job to prevent regressions like this in the
future.
For the record though, I'm listing alternative approaches I thought of:
- We could get rid of the debug/release distinction and just add `@Todo`
type metadata everywhere. This has possible affects on runtime. The main
reason I didn't follow through with this is that the size of `Type`
increases. We would either have to adapt the `assert_eq_size!` test or
get rid of it. Even if we add messages everywhere and get rid of the
file-and-line-variant in the enum, it's not enough to get back to the
current release-mode size of `Type`.
- We could generally discard `@Todo` meta information when using it in
tests. I think this would be a huge drawback. I like that we can have
the actual messages in the mdtest. And make sure we get the expected
`@Todo` type, not just any `@Todo`. It's also helpful when debugging
tests.
closes#14594
## Test Plan
```rs
cargo nextest run --release
```
We should probably get rid of this entirely and subsume it's
functionality in the normal ecosystem checks? I don't think we're using
the black comparison tests anymore, but maybe someone wants it?
There are a few major parts to this:
1. Making the formatter script idempotent, so it can be run repeatedly
and is robust to changing commits
2. Reducing the overhead of the git operations, minimizing the data
transfer
3. Parallelizing all the git operations by repository
This reduces the setup time from 80s to 16s (locally).
The initial motivation for idempotency was to include the repositories
in the GitHub Actions cache. I'm not sure it's worth it yet — they're
about 1GB and would consume our limited cache space. Regardless, it
improves correctness for local invocations.
The total runtime of the job is reduced from ~4m to ~3m.
I also made some cosmetic changes to the output paths and such.
This is one of the slowest remaining jobs in the pull request CI. We
could use a larger runner for a trivial speed-up (in exchange for $$),
but I don't think this is going to break often enough to merit testing
on every pull request commit? It's not a required job, so I don't feel
strongly about it, but it feels like a bit of a waste of compute.
Originally added in https://github.com/astral-sh/ruff/pull/11182
Reduces Linux test CI to 1m 40s (16 core) or 2m 56s (8 core) to from 4m
25s. Times are approximate, as runner performance is pretty variable.
In uv, we use the 16 core runners.
## Summary
This PR updates `ruff` to match `uv` updated [docker releases
approach](https://github.com/astral-sh/uv/blob/main/.github/workflows/build-docker.yml).
It's a combined PR with changes from these PR's
* https://github.com/astral-sh/uv/pull/6053
* https://github.com/astral-sh/uv/pull/6556
* https://github.com/astral-sh/uv/pull/6734
* https://github.com/astral-sh/uv/pull/7568
Summary of changes / features
1. This change would publish an additional tags that includes only
`major.minor`.
For a release with `x.y.z`, this would publish the tags:
* ghcr.io/astral-sh/ruff:latest
* ghcr.io/astral-sh/ruff:x.y.z
* ghcr.io/astral-sh/ruff:x.y
2. Parallelizes multi-platform builds using multiple workers (hence the
new docker-build / docker-publish jobs), which cuts docker releases time
in half.
3. This PR introduces additional images with the ruff binaries from
scratch for both amd64/arm64 and makes the mapping easy to configure by
generating the Dockerfile on the fly. This approach focuses on
minimizing CI time by taking advantage of dedicating a worker per
mapping (20-30s~ per job). For example, on release `x.y.z`, this will
publish the following image tags with format
`ghcr.io/astral-sh/ruff:{tag}` with manifests for both amd64/arm64. This
also include `x.y` tags for each respective additional tag. Note, this
version does not include the python based images, unlike `uv`.
* From **scratch**: `latest`, `x.y.z`, `x.y` (currently being published)
* From **alpine:3.20**: `alpine`, `alpine3.20`, `x.y.z-alpine`,
`x.y.z-alpine3.20`
* From **debian:bookworm-slim**: `debian-slim`, `bookworm-slim`,
`x.y.z-debian-slim`, `x.y.z-bookworm-slim`
* From **buildpack-deps:bookworm**: `debian`, `bookworm`,
`x.y.z-debian`, `x.y.z-bookworm`
4. This PR also fixes `org.opencontainers.image.version` for all tags
(including the one from `scratch`) to contain the right release version
instead of branch name `main` (current behavior).
```
> docker inspect ghcr.io/astral-sh/ruff:0.6.4 | jq -r
'.[0].Config.Labels'
{
...
"org.opencontainers.image.version": "main"
}
```
Closes https://github.com/astral-sh/ruff/issues/13481
## Test Plan
Approach mimics `uv` with almost no changes so risk is low but I still
tested the full workflow.
* I have a working CI release pipeline on my fork run
https://github.com/samypr100/ruff/actions/runs/10966657733
* The resulting images were published to
https://github.com/samypr100/ruff/pkgs/container/ruff
## Summary
Adds a markdown-based test framework for writing tests of type inference
and type checking. Fixes#11664.
Implements the basic required features. A markdown test file is a suite
of tests, each test can contain one or more Python files, with
optionally specified path/name. The test writes all files to an
in-memory file system, runs red-knot, and matches the resulting
diagnostics against `Type: ` and `Error: ` assertions embedded in the
Python source as comments.
We will want to add features like incremental tests, setting custom
configuration for tests, writing non-Python files, testing syntax
errors, capturing full diagnostic output, etc. There's also plenty of
room for improved UX (colored output?).
## Test Plan
Lots of tests!
Sample of the current output when a test fails:
```
Running tests/inference.rs (target/debug/deps/inference-7c96590aa84de2a4)
running 1 test
test inference::path_1_resources_inference_numbers_md ... FAILED
failures:
---- inference::path_1_resources_inference_numbers_md stdout ----
inference/numbers.md - Numbers - Floats
/src/test.py
line 2: unexpected error: [invalid-assignment] "Object of type `Literal["str"]` is not assignable to `int`"
thread 'inference::path_1_resources_inference_numbers_md' panicked at crates/red_knot_test/src/lib.rs:60:5:
Some tests failed.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
inference::path_1_resources_inference_numbers_md
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s
error: test failed, to rerun pass `-p red_knot_test --test inference`
```
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
This PR changes removes the typeshed stubs from the vendored file system
shipped with ruff
and instead ships an empty "typeshed".
Making the typeshed files optional required extracting the typshed files
into a new `ruff_vendored` crate. I do like this even if all our builds
always include typeshed because it means `red_knot_python_semantic`
contains less code that needs compiling.
This also allows us to use deflate because the compression algorithm
doesn't matter for an archive containing a single, empty file.
## Test Plan
`cargo test`
I verified with ` cargo tree -f "{p} {f}" -p <package> ` that:
* red_knot_wasm: enables `deflate` compression
* red_knot: enables `zstd` compression
* `ruff`: uses stored
I'm not quiet sure how to build the binary that maturin builds but
comparing the release artifact size with `strip = true` shows a `1.5MB`
size reduction
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Make `cargo doc -p red_knot_python_semantic --document-private-items`
run warning-free. I'd still like to do this for all of ruff and start
enforcing it in CI (https://github.com/astral-sh/ruff/issues/12372) but
haven't gotten to it yet. But in the meantime I'm trying to maintain it
for at least `red_knot_python_semantic`, as it helps to ensure our doc
comments stay up to date.
A few of the comments I just removed or shortened, as their continued
relevance wasn't clear to me; please object in review if you think some
of them are important to keep!
Also remove a no-longer-needed `allow` attribute.
## Summary
These are now `post-announce-jobs`. So if they fail, the release itself
will still succeed, which seems ok. (If we make them `publish-jobs`,
then we might end up publishing to PyPI but failing the release itself
if one of these fails.)
The intent is that these are still runnable via `workflow_dispatch` too.
Closes https://github.com/astral-sh/ruff/issues/12074.
Micha Reiser
renovate[bot]
David Peter
samypr100
Douglas Creager
Brent Westbrook
Shaygan Hooshyari
Zanie Blue
Dylan
Alex Waygood
Dhruv Manilawala
Connor Skees
Steve C
Carl Meyer
Charlie Marsh
Matthew Runyon