test(fuzz): Enhance fuzzing capabilities and CI workflows with documentation#10
test(fuzz): Enhance fuzzing capabilities and CI workflows with documentation#10mergify[bot] merged 19 commits intomainfrom
Conversation
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…ments Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
… compatibility Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
|
Caution Review failedFailed to post review comments Summary by CodeRabbit
WalkthroughAdds fuzzing and compatibility CI, lowers MSRV to 1.85, exposes an internal read_bounded for fuzzing, adds cap-probing logic to read_bounded, introduces a fuzz workspace with two targets, and adds proptest property tests and related documentation and config updates. Changes
Sequence Diagram(s)sequenceDiagram
participant GitHub as "GitHub Actions"
participant Runner as "Runner (ubuntu-latest)"
participant Checkout as "checkout@v4"
participant Toolchain as "rustup / rust-toolchain"
participant Cargo as "cargo / cargo-fuzz"
participant Artifact as "artifact storage"
Note over GitHub: Trigger: schedule / PR / manual
GitHub->>Runner: start job (compat / fuzz)
Runner->>Checkout: checkout code (pin tag)
Runner->>Toolchain: install toolchain (matrix)
Toolchain->>Cargo: build / test or install cargo-fuzz
Cargo->>Cargo: run cargo build/test OR cargo fuzz run <target>
alt fuzz found crash
Cargo->>Artifact: upload crash artifacts
Artifact->>GitHub: artifact available (30d)
end
Runner->>GitHub: job status (success/neutral/failure)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 📃 Configuration Change RequirementsWonderful, this rule succeeded.Mergify configuration change
🟢 CI must passWonderful, this rule succeeded.All CI checks must pass. Release-plz PRs are exempt because they only bump versions and changelogs (code was already tested on main), and GITHUB_TOKEN-triggered force-pushes suppress CI.
🟢 Do not merge outdated PRsWonderful, this rule succeeded.Make sure PRs are within 10 commits of the base branch before merging
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive fuzzing and property-based testing infrastructure to the mmap-guard crate, along with CI workflows and documentation. It exposes an internal function (read_bounded) behind a feature flag for fuzz targets, adds two cargo-fuzz targets, two proptest test suites, and two new CI workflows (weekly fuzzing and Rust version compatibility matrix).
Changes:
- Added
fuzz/workspace with two coverage-guided fuzz targets (fuzz_read_bounded,fuzz_map_file),__fuzzfeature flag for internal API exposure, andproptest-based property tests for bothmap_fileandread_bounded. - Added
.github/workflows/fuzz.ymland.github/workflows/compat.ymlCI workflows with merge queue gating via Mergify'scheck-success-or-neutral, plus local CI simulation recipes injustfileusingact. - Lowered MSRV from 1.89 to 1.85 (edition 2024 minimum), updated documentation (
AGENTS.md,GOTCHAS.md,docs/src/testing.md), bumped tool versions inmise.toml/mise.lock, and added fuzz artifact paths to.gitignore.
Reviewed changes
Copilot reviewed 15 out of 17 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
fuzz/Cargo.toml |
New fuzz workspace with libfuzzer-sys, tempfile, and __fuzz feature dependency on mmap-guard |
fuzz/fuzz_targets/fuzz_read_bounded.rs |
Structured fuzz target for read_bounded with Arbitrary-derived input |
fuzz/fuzz_targets/fuzz_map_file.rs |
Fuzz target for map_file round-trip integrity via temp files |
src/lib.rs |
Conditional #[doc(hidden)] re-export of read_bounded under __fuzz feature |
src/load.rs |
Changed read_bounded to pub, added clippy allow attributes, added proptest property tests |
tests/prop_map_file.rs |
Integration property test for map_file round-trip and empty file rejection |
Cargo.toml |
MSRV lowered to 1.85, added __fuzz feature, added proptest dev-dependency, excluded /fuzz from package |
.github/workflows/fuzz.yml |
Weekly nightly fuzzing workflow with crash artifact upload |
.github/workflows/compat.yml |
Weekly Rust version compatibility matrix (stable, stable-2, stable-5, MSRV) |
.mergify.yml |
Added check-success-or-neutral merge conditions for fuzz and compat checks |
justfile |
Added act-based local CI simulation recipes (dry-run, run, per-workflow, per-job) |
AGENTS.md |
Documented fuzzing targets, property tests, __fuzz feature, and CI workflows |
GOTCHAS.md |
Documented fuzzing caveats, act gotchas, and action SHA verification |
docs/src/testing.md |
Updated test organization table to include load.rs tests |
.gitignore |
Added fuzz/artifacts/ and fuzz/corpus/ |
mise.toml |
Bumped release-plz to 0.3.157 |
mise.lock |
Reformatted TOML and updated tool checksums/versions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Related Documentation 6 document(s) may need updating based on files changed in this PR: DaemonEye CI and Automation HooksView Suggested Changes@@ -8,6 +8,8 @@
- `docs.yml`: Documentation build and deployment to GitHub Pages ([docs.yml](https://github.com/EvilBit-Labs/DaemonEye/blob/a169b6886897bf281c27ce9d86d4738233658cd4/.github/workflows/docs.yml)).
- `codeql.yml`: Code analysis and security scanning ([codeql.yml](https://github.com/EvilBit-Labs/DaemonEye/blob/a169b6886897bf281c27ce9d86d4738233658cd4/.github/workflows/codeql.yml)).
- `scorecard.yml`: Supply-chain security analysis using OpenSSF Scorecard ([scorecard.yml](https://github.com/EvilBit-Labs/DaemonEye/blob/main/.github/workflows/scorecard.yml)).
+- `fuzz.yml`: Coverage-guided fuzzing with cargo-fuzz on nightly Rust.
+- `compat.yml`: Compatibility testing across multiple Rust toolchain versions.
**Scorecard Supply-Chain Security Workflow**
@@ -20,6 +22,28 @@
- Supports optional configuration for private repositories and branch protection checks.
This workflow helps maintainers monitor supply-chain security posture and enables consumers to access Scorecard results and badges. For details on configuration and authentication, see the [Scorecard Action documentation](https://github.com/ossf/scorecard-action).
+
+**Fuzz Workflow**
+
+The `fuzz.yml` workflow runs coverage-guided fuzzing using `cargo-fuzz` on nightly Rust. It is triggered by a weekly schedule (Monday 06:00 UTC), pull requests targeting the `main` branch, and `workflow_dispatch` for manual runs. The workflow uses conditional execution: it only runs on merge queue PRs (via `startsWith(github.head_ref, 'mergify/merge-queue/')`) to avoid consuming CI resources on regular PRs.
+
+The workflow uses a matrix strategy with `fail-fast: false` over two fuzz targets:
+- `fuzz_read_bounded`: Tests bounded read operations with structured input.
+- `fuzz_map_file`: Tests round-trip mapping integrity for file operations.
+
+Each target runs for 60 seconds by default (configurable via the `FUZZ_SECONDS` environment variable). The workflow installs the nightly toolchain and `cargo-fuzz`, executes each fuzz target, and uploads crash artifacts on failure with 30-day retention. This enables maintainers to review crashes discovered during fuzzing runs.
+
+**Compatibility Workflow**
+
+The `compat.yml` workflow tests the project across multiple Rust toolchain versions to ensure compatibility. It is triggered by a weekly schedule (Monday 06:00 UTC), pull requests targeting the `main` branch, and `workflow_dispatch` for manual runs. Like the fuzz workflow, it uses conditional execution to run only on merge queue PRs.
+
+The workflow uses a matrix strategy with `fail-fast: false` over the following toolchains:
+- `stable`: The latest stable Rust release.
+- `stable minus 2 releases`: Compatibility with recent prior stable versions.
+- `stable minus 5 releases`: Broader backward compatibility testing.
+- `1.85.0`: The minimum supported Rust version (MSRV), which is the edition 2024 minimum.
+
+For each toolchain, the workflow runs `cargo build` and `cargo test` with default features to validate that the project compiles and passes tests across supported Rust versions.
**Workflow Triggers and Automation Hooks**
@@ -63,6 +87,10 @@
The project uses Mergify (`.mergify.yml`) to automate handling of bot-generated pull requests, streamlining dependency updates and automated maintenance PRs. Mergify automatically approves and merges PRs from Dependabot, Dosubot, and release-plz after CI checks pass, and keeps these PRs up to date with the main branch.
+Both the fuzz and compatibility workflows integrate with Mergify's merge protections using `check-success-or-neutral` conditions. This two-step CI pattern allows the workflows to be skipped on regular PRs (reducing CI resource consumption), but enforces that they must pass if they run in the merge queue. If either workflow fails during a merge queue run, the merge is blocked. The Mergify configuration includes checks for all matrix jobs in both workflows:
+- Fuzz workflow: `fuzz (fuzz_read_bounded)` and `fuzz (fuzz_map_file)`
+- Compatibility workflow: `compat (stable)`, `compat (stable minus 2 releases)`, `compat (stable minus 5 releases)`, and `compat (1.85.0)`
+
**Maintaining and Extending Automation Workflows**
To maintain and extend the automation workflows:❌ Declined mmap-guard CI Pipeline And Cross-Platform TestingView Suggested Changes@@ -13,6 +13,10 @@
### Overview and Triggers
[The main CI workflow defined in `.github/workflows/ci.yml` runs on every push to main and on all pull requests](https://app.dosu.dev/documents/2ac8341a-733e-4e85-bb1d-750a84dccb16). The workflow consists of four distinct jobs that execute sequentially with dependencies.
+
+Additional specialized workflows run on weekly schedules and merge queue events:
+- **`.github/workflows/fuzz.yml`** — Coverage-guided fuzz testing with cargo-fuzz
+- **`.github/workflows/compat.yml`** — Rust version compatibility testing across stable, stable-2, stable-5, and MSRV
### Job 1: Quality
@@ -66,6 +70,34 @@
- **SHA-pinned GitHub Actions** — Instead of `actions/checkout@v3`, uses full SHA like `actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd` to prevent tag-mutation attacks
- **Minimal permissions** — `permissions: contents: read` by default (least privilege)
- **Concurrency control** — `cancel-in-progress: true` setting prevents resource waste
+
+### Fuzz Workflow (fuzz.yml)
+
+The fuzz workflow performs coverage-guided fuzzing to discover edge cases and potential panics:
+
+- **Triggers**: Weekly schedule (Mondays 06:00 UTC), merge queue events, manual dispatch
+- **Toolchain**: Requires nightly Rust for cargo-fuzz
+- **Fuzz Targets**: Matrix strategy tests both `fuzz_read_bounded` and `fuzz_map_file` targets
+- **Duration**: Each target runs for 60 seconds (`FUZZ_SECONDS=60`)
+- **Artifacts**: Automatically uploads crash artifacts with 30-day retention on failure
+- **Gating**: Uses `check-success-or-neutral` in merge protection to allow skipped scheduled runs
+
+The workflow only executes on merge queue PRs or scheduled runs, not on regular pull requests. This design avoids blocking every PR with long-running fuzz tests while still validating changes before merge.
+
+### Compatibility Workflow (compat.yml)
+
+The compatibility workflow validates behavior across Rust versions:
+
+- **Triggers**: Weekly schedule (Mondays 06:00 UTC), merge queue events, manual dispatch
+- **Rust Versions**: Matrix tests across:
+ - `stable` — Latest stable release
+ - `stable minus 2 releases` — Two releases behind
+ - `stable minus 5 releases` — Five releases behind
+ - `1.85.0` — MSRV (Minimum Supported Rust Version for edition 2024)
+- **Jobs**: Build and test execution for each version
+- **Gating**: Uses `check-success-or-neutral` in merge protection to allow skipped scheduled runs
+
+Like the fuzz workflow, this only runs on merge queue PRs or weekly schedules to avoid blocking regular development while ensuring compatibility before merge.
## Platform-Specific Testing
@@ -146,6 +178,15 @@
**Important configuration note:** [Mergify evaluates from the main branch configuration, meaning changes to Mergify rules in feature branches won't take effect until merged to main](https://app.dosu.dev/documents/4a2f4809-90bd-4455-8ea1-c7f927f66ce4).
+Merge conditions include:
+- **Core CI workflows**: All jobs from the main CI pipeline must pass (quality, test, test-cross-platform, coverage)
+- **Weekly workflows**: Fuzz and compatibility checks use `check-success-or-neutral` conditions, which:
+ - Block merge if the workflow ran and failed
+ - Allow merge if the workflow didn't run (neutral status)
+ - Gate merge queue PRs but not regular PRs
+
+This design ensures weekly fuzz and compatibility tests validate changes entering the merge queue without blocking every PR during regular development.
+
Release workflow:
1. **Release PR Creation**: release-plz creates a PR with version bumps and CHANGELOG
2. **Automated Merge**: [Mergify automatically merges the PR after DCO checks pass](https://app.dosu.dev/documents/0e7b2ef2-fec8-41e2-af4c-7ece10a919d3)
@@ -195,6 +236,17 @@
[Based on background context, mise-managed cargo subcommands must be invoked as standalone binaries](https://app.dosu.dev/documents/4a2f4809-90bd-4455-8ea1-c7f927f66ce4) (e.g., `dist plan` not `cargo dist plan`).
+### Local Workflow Simulation with act
+
+The justfile provides recipes for testing GitHub Actions workflows locally using `act`:
+
+- `just act-run <workflow>` — Run a specific workflow locally
+- `just act-dry-run <workflow>` — Dry-run a workflow to validate syntax and steps
+- `just act-run-workflow <workflow> [event]` — Run workflow with custom event trigger
+- `just act-run-job <workflow> <job> [event]` — Run specific job from a workflow
+
+The `lint-actions` command validates all workflow files including `compat.yml` and `fuzz.yml` using actionlint.
+
### Pre-commit Hook Behavior
[Pre-commit hooks run automatically on `git commit`](https://app.dosu.dev/documents/cc55aa67-9430-4d02-9574-59b9b3651de4):
@@ -280,7 +332,9 @@
| File Path | Purpose | Key Features |
|-----------|---------|--------------|
| `.github/workflows/ci.yml` | Main CI pipeline | quality, test, test-cross-platform, coverage jobs |
-| `.mergify.yml` | Merge protection rules | Auto-merge configuration for DCO verification |
+| `.github/workflows/fuzz.yml` | Fuzzing workflow | Weekly coverage-guided fuzzing with cargo-fuzz, nightly toolchain |
+| `.github/workflows/compat.yml` | Compatibility testing | Weekly validation across Rust versions (stable, stable-2, stable-5, MSRV) |
+| `.mergify.yml` | Merge protection rules | Auto-merge configuration with check-success-or-neutral for weekly workflows |
| `Cargo.toml` | Lint configuration | `[workspace.lints.clippy]` with pedantic/nursery/cargo groups |
| `src/lib.rs` | Crate-level configuration | `#![deny(clippy::undocumented_unsafe_blocks)]` |
| `src/map.rs` | Test organization | Successful mapping, empty file rejection, missing file tests |
@@ -288,7 +342,7 @@
| `src/file_data.rs` | Test organization | `Deref`/`AsRef` impls, empty variant tests |
| `mise.toml` | Tool management | Development tool versions and installation |
| `.pre-commit-config.yaml` | Pre-commit hooks | Format, lint, audit, and security checks |
-| `justfile` | Task automation | `ci-check`, `coverage-check`, `audit`, `deny` recipes |
+| `justfile` | Task automation | `ci-check`, `coverage-check`, `audit`, `deny`, `act-run` recipes |
| `deny.toml` | Dependency policies | License allowlist, banned crates, source restrictions |
## Related TopicsCI/CD Pipeline SecurityView Suggested Changes@@ -118,13 +118,23 @@
### Security Workflows
-Three specialized security workflows run on scheduled intervals:
+Five specialized security and quality workflows run on scheduled intervals:
- **audit.yml** — Daily execution of cargo-audit with automatic issue creation
- **security.yml** — Daily cargo-deny and cargo-outdated checks
- **scorecard.yml** — Weekly OSSF Scorecard assessment
+- **fuzz.yml** — Weekly coverage-guided fuzz testing with cargo-fuzz
+- **compat.yml** — Weekly compatibility testing across Rust toolchain versions
These workflows operate independently of the main CI pipeline, ensuring security monitoring continues even when no code changes are being made.
+
+#### Fuzzing Workflow
+
+The `fuzz.yml` workflow performs coverage-guided fuzz testing to discover edge cases and potential security vulnerabilities. Running weekly and on merge queue events, it executes two fuzz targets (`fuzz_read_bounded` and `fuzz_map_file`) for 60 seconds each using cargo-fuzz on nightly Rust. The workflow implements a matrix strategy to parallelize execution, with minimal read-only permissions and automatic artifact upload for any discovered crashes. Merge protection uses `check-success-or-neutral` conditions, allowing merges when the workflow is skipped on scheduled runs while still gating merges when triggered by pull requests.
+
+#### Compatibility Workflow
+
+The `compat.yml` workflow validates functionality across multiple Rust toolchain versions to ensure broad compatibility. Running weekly and on merge queue events, it tests against stable, stable minus 2 releases, stable minus 5 releases, and the MSRV (Minimum Supported Rust Version) of 1.85.0. The workflow executes the full build and test suite on each toolchain version with a fail-fast: false strategy to collect results from all versions. Like the fuzzing workflow, it uses `check-success-or-neutral` merge protection to allow optional scheduled runs while gating merges when triggered.
### Local CI Parity
@@ -159,6 +169,10 @@
The release workflow represents a critical exception to this pattern. The `release-plz-pr` job sets `cancel-in-progress: false` to prevent race conditions during release preparation, ensuring that release version calculations and changelog generation complete atomically even if new commits arrive during execution.
+### Merge Queue Protection
+
+The `.mergify.yml` configuration implements sophisticated merge protection that accommodates both required and optional CI workflows. Core workflows (quality, test, coverage) use `check-success` conditions that require explicit passing status before merge. Weekly workflows (fuzz, compat) use `check-success-or-neutral` conditions that allow merges when checks are skipped (neutral status on scheduled runs) while still blocking merges if the workflows run and fail. This design ensures that optional scheduled workflows don't block routine development while still providing gate protection when triggered by pull requests or merge queue events.
+
### Credential Security
Workflow checkout operations use `persist-credentials: false` for security-sensitive operations, particularly in the OSSF Scorecard workflow. This prevents GitHub credentials from being persisted to the local Git configuration, reducing the risk of credential exposure if subsequent workflow steps are compromised.
@@ -330,6 +344,8 @@
| `.github/workflows/security.yml` | Daily security monitoring | cargo-deny policy checks, cargo-outdated analysis |
| `.github/workflows/scorecard.yml` | Supply chain assessment | Weekly OSSF Scorecard runs, SARIF upload to code-scanning |
| `.github/workflows/audit.yml` | Vulnerability scanning | Daily cargo-audit execution, automatic issue creation |
+| `.github/workflows/fuzz.yml` | Weekly fuzz testing | Coverage-guided fuzzing with cargo-fuzz, crash artifact upload |
+| `.github/workflows/compat.yml` | Rust version compatibility | Testing across stable, MSRV, and historical toolchains |
| `.github/workflows/release-plz.yml` | Automated releases | Version bumping, changelog generation, crates.io publication |
| `.github/dependabot.yml` | Dependency automation | Weekly updates for Cargo, GitHub Actions, DevContainers |
| `deny.toml` | Dependency policies | License allowlist, banned crates, source restrictions |✅ Accepted CONTRIBUTING
|
|
@Mergifyio queue |
Merge Queue StatusRule:
This pull request spent 4 minutes 45 seconds in the queue, including 4 minutes 33 seconds running CI. Required conditions to merge
|
This pull request introduces coverage-guided fuzz testing and property-based testing to the project, along with new CI workflows to ensure ongoing compatibility and fuzz safety. It adds a dedicated fuzz workspace, exposes internal APIs for fuzzing, and updates documentation and CI configuration to support these enhancements.
Fuzzing and Property-Based Testing
fuzz/workspace with two fuzz targets:fuzz_read_bounded(structured input for bounded reads) andfuzz_map_file(round-trip mapping integrity), usingcargo-fuzzand nightly Rust. [1] [2] [3]__fuzzfeature flag to expose the internalread_boundedfunction for fuzz targets, with appropriate visibility and documentation warnings. (Ff803458L139R139, [1] [2]proptestas a dev-dependency and documented property tests for bothmap_fileandread_bounded. (Ff803458L139R139, AGENTS.mdL73-R113)Continuous Integration and Merge Protections
.github/workflows/fuzz.ymland.github/workflows/compat.ymlfor weekly fuzzing and compatibility checks across Rust versions, including merge queue gating and artifact uploads for fuzz crashes. [1] [2].mergify.ymlto gate merges on the results of fuzz and compatibility workflows, allowing merges only if checks are successful or neutral.justfilewith new recipes for running and dry-running all workflows usingact, including schedule-only workflows.Documentation and Configuration Updates
AGENTS.mdandGOTCHAS.mdwith detailed instructions and caveats for fuzzing, property tests, CI workflow triggers, and internal API exposure. [1] [2] [3]Cargo.tomlto lower MSRV to 1.85, add the__fuzzfeature, and includeproptestas a dev-dependency. (Cargo.tomlL15-R23, Ff803458L139R139)justfile,mise.toml, and other docs for linting, testing, and tool version bumps. [1] [2] [3]References: [1] [2] [3] Ff803458L139R139, [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]