-
Notifications
You must be signed in to change notification settings - Fork 304
perf: add and harden synthetic scale-test suite #3038
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Arths17
wants to merge
6
commits into
facebook:main
Choose a base branch
from
Arths17:perf-stress-suite
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
8eabe71
perf: add synthetic scale-test suite for indexing and recursion stress
Arths17 9194095
perf: harden stress-test transact typing and public usage docs
Arths17 c440422
modified: tests/perf/scale_test/__pycache__/generate_stress_test.cp…
Arths17 1545937
deleted: tests/perf/scale_test/__pycache__/__init__.cpython-312.pyc
Arths17 e93f3c9
perf: sync generated exports and document Windows venv setup
Arths17 64f4953
Merge branch 'main' into perf-stress-suite
Arths17 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Synthetic Performance Tests | ||
|
|
||
| This directory contains synthetic Python workloads used to stress-test Pyrefly indexing and type-solving throughput. | ||
|
|
||
| ## Structure | ||
|
|
||
| - `perf/scale_test/`: Generated "synthetic enterprise" package with dense type annotations, deep inheritance, and import meshes. | ||
|
|
||
| ## Scale Test Goals | ||
|
|
||
| The scale test workload is designed to apply pressure to: | ||
|
|
||
| 1. Deep recursive solving paths in inheritance hierarchies. | ||
| 2. Generic and protocol-heavy type relationships. | ||
| 3. Inter-module dependency graph construction. | ||
| 4. File-volume and line-volume indexing throughput. | ||
|
|
||
| ## Generator | ||
|
|
||
| The scale test modules are produced by: | ||
|
|
||
| - `perf/scale_test/generate_stress_test.py` | ||
|
|
||
| Requirements: | ||
|
|
||
| - Python 3.12+ available as `python3` (macOS/Linux) or `py -3` (Windows). | ||
| - Run commands from the repository root. | ||
|
|
||
| Optional virtual environment setup: | ||
|
|
||
| ```bash | ||
| python3 -m venv .venv | ||
| source .venv/bin/activate | ||
| python3 -m pip install --upgrade pip | ||
| ``` | ||
|
|
||
| Windows PowerShell equivalent: | ||
|
|
||
| ```powershell | ||
| py -3 -m venv .venv | ||
| .\.venv\Scripts\Activate.ps1 | ||
| py -3 -m pip install --upgrade pip | ||
| ``` | ||
|
|
||
| Run from repository root: | ||
|
|
||
| ```bash | ||
| python3 tests/perf/scale_test/generate_stress_test.py --core-only | ||
| ``` | ||
|
|
||
| Windows PowerShell equivalent: | ||
|
|
||
| ```powershell | ||
| py -3 tests/perf/scale_test/generate_stress_test.py --core-only | ||
| ``` | ||
|
|
||
| Generate full suite (default: 120 modules, min 520 lines each): | ||
|
|
||
| ```bash | ||
| python3 tests/perf/scale_test/generate_stress_test.py | ||
| ``` | ||
|
|
||
| Windows PowerShell equivalent: | ||
|
|
||
| ```powershell | ||
| py -3 tests/perf/scale_test/generate_stress_test.py | ||
| ``` | ||
|
|
||
| Customize generation: | ||
|
|
||
| ```bash | ||
| python3 tests/perf/scale_test/generate_stress_test.py \ | ||
| --module-count 150 \ | ||
| --min-lines 700 | ||
| ``` | ||
|
|
||
| Windows PowerShell equivalent: | ||
|
|
||
| ```powershell | ||
| py -3 tests/perf/scale_test/generate_stress_test.py --module-count 150 --min-lines 700 | ||
| ``` | ||
|
|
||
| ## Expected Core Outputs | ||
|
|
||
| Running with `--core-only` emits the initial five baseline modules: | ||
|
|
||
| - `base_types.py` | ||
| - `data_layer.py` | ||
| - `service_mesh.py` | ||
| - `domain_models.py` | ||
| - `orchestration.py` | ||
|
|
||
| All generated files are valid Python and intentionally large to provide stable stress-testing inputs. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| """Synthetic enterprise stress-test package for Pyrefly.""" | ||
|
|
||
| from . import base_types | ||
| from . import data_layer | ||
| from . import service_mesh | ||
| from . import domain_models | ||
| from . import orchestration | ||
|
|
||
| __all__ = [ | ||
| "base_types", | ||
| "data_layer", | ||
| "service_mesh", | ||
| "domain_models", | ||
| "orchestration" | ||
| ] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.