Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use uv run to run the Python-based fuzzer #13074

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,6 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- uses: actions/download-artifact@v4
name: Download Ruff binary to test
id: download-cached-binary
Expand All @@ -307,7 +305,7 @@ jobs:
# Make executable, since artifact download doesn't preserve this
chmod +x ${{ steps.download-cached-binary.outputs.download-path }}/ruff

python scripts/fuzz-parser/fuzz.py 0-500 --test-executable ${{ steps.download-cached-binary.outputs.download-path }}/ruff
uv run scripts/fuzz-parser.py 0-500 --test-executable ${{ steps.download-cached-binary.outputs.download-path }}/ruff

scripts:
name: "test scripts"
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/daily_fuzz.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ jobs:
python-version: "3.12"
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
Expand All @@ -49,7 +47,7 @@ jobs:
# but this is outweighed by the fact that a release build takes *much* longer to compile in CI
run: cargo build --locked
- name: Fuzz
run: python scripts/fuzz-parser/fuzz.py $(shuf -i 0-9999999999999999999 -n 1000) --test-executable target/debug/ruff
run: uv run scripts/fuzz-parser.py $(shuf -i 0-9999999999999999999 -n 1000) --test-executable target/debug/ruff

create-issue-on-failure:
name: Create an issue if the daily fuzz surfaced any bugs
Expand Down
11 changes: 5 additions & 6 deletions crates/ruff_python_parser/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,19 +58,18 @@ cargo test --package ruff_python_parser
The Ruff project includes a Python-based fuzzer that can be used to run the parser on
randomly generated (but syntactically valid) Python source code files.

To run the fuzzer, first install the required dependencies:
To run the fuzzer, first [install `uv`](https://docs.astral.sh/uv/getting-started/installation/):

```sh
uv pip install -r scripts/fuzz-parser/requirements.txt
```
- Unix: `curl -LsSf https://astral.sh/uv/install.sh | sh`
- Windows: `powershell -c "irm https://astral.sh/uv/install.ps1 | iex"`

Then, run the fuzzer with the following command:

```sh
python scripts/fuzz-parser/fuzz.py
uv run scripts/fuzz-parser.py
```

Refer to the [fuzz.py](https://github.com/astral-sh/ruff/blob/main/scripts/fuzz-parser/fuzz.py)
Refer to the [fuzz-parser.py](https://github.com/astral-sh/ruff/blob/main/scripts/fuzz-parser.py)
script for more information or use the `--help` flag to see the available options.

#### CI
Expand Down
26 changes: 26 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,29 @@ version_files = [
"crates/ruff_wasm/Cargo.toml",
"scripts/benchmarks/pyproject.toml",
]

[tool.uv]
# Dependencies for our various Python scripts in `scripts/`.
# These could possibly use inline script metadata, but then the exact
# version of the dependency installed wouldn't be recorded in `uv.lock`.
dev-dependencies = [
# Required by `generate_known_standard_library.py`
"stdlibs>=2024.1.28",
# Required by `fuzz-parser.py
"pysource-codegen",
"pysource-minimize",
"rich-argparse",
"ruff",
"termcolor",
# Docs dependencies are deliberately omitted,
# as they're unfortunately quite complicated.
# See `docs/requirements.txt` and `docs/requirements-insiders.txt`
# for the requirements of `generate_mkdocs.py`, `_mdformat_utils.py`
# and `check_docs_formatted.py`
]

# No need to build Ruff from source just to run our development scripts
package = false

[tool.uv.workspace]
members = []
13 changes: 6 additions & 7 deletions scripts/fuzz-parser/fuzz.py → scripts/fuzz-parser.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
"""
Run the parser on randomly generated (but syntactically valid) Python source-code files.

To install all dependencies for this script into an environment using `uv`, run:
uv pip install -r scripts/fuzz-parser/requirements.txt

Example invocations of the script:
Example invocations of the script using `uv` from the project root
(which will automatically run the script in an environment
with all dependencies installed):
- Run the fuzzer using seeds 0, 1, 2, 78 and 93 to generate the code:
`python scripts/fuzz-parser/fuzz.py 0-2 78 93`
`uv run scripts/fuzz-parser.py 0-2 78 93`
- Run the fuzzer concurrently using seeds in range 0-10 inclusive,
but only reporting bugs that are new on your branch:
`python scripts/fuzz-parser/fuzz.py 0-10 --new-bugs-only`
`uv run scripts/fuzz-parser.py 0-10 --new-bugs-only`
- Run the fuzzer concurrently on 10,000 different Python source-code files,
using a random selection of seeds, and only print a summary at the end
(the `shuf` command is Unix-specific):
`python scripts/fuzz-parser/fuzz.py $(shuf -i 0-1000000 -n 10000) --quiet
`uv run scripts/fuzz-parser.py $(shuf -i 0-1000000 -n 10000) --quiet
"""

from __future__ import annotations
Expand Down
5 changes: 0 additions & 5 deletions scripts/fuzz-parser/requirements.in

This file was deleted.

36 changes: 0 additions & 36 deletions scripts/fuzz-parser/requirements.txt

This file was deleted.

8 changes: 3 additions & 5 deletions scripts/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[project]
name = "scripts"
version = "0.0.1"
dependencies = ["stdlibs"]
requires-python = ">=3.8"
# This `pyproject.toml` file provides linting configuration for our Python scripts
# It doesn't list dev dependencies requried by those scripts;
# those are listed in the `pyproject.toml` file in the project root.

[tool.black]
line-length = 88
Expand Down
Loading
Loading