feat: add --mode and --operators flags to mutator and testing tool
#116
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements configurable mutation operator modes (light, medium, heavy) based on operator effectiveness analysis, which is at the end of the description below, (22,597 mutants across Aptos' Move framework). This allows users to trade off speed vs thoroughness when running mutation tests. It also allows to add different operators that we support, per operator basis. Changes applied to the
move-mutatorandmove-mutation-testCLI tools.Motivation
Running full mutation testing with all operators can be time-consuming.
This PR enables faster feedback loops by allowing users to run only the most effective operators or custom ones that they want to test for.
Changes
CLI Additions
Both
move-mutatorandmove-mutation-testnow support:--mode <light|medium|heavy>- Predefined operator setslight: binary_operator_swap, break_continue_replacement, delete_statementmedium: light + literal_replacementmedium-only- only the operator added in medium (not including light)heavy(default): allheavy-only- only the operators added in heavy (not including light/medium)--operators <op1,op2,...>- Custom operator selection--modeUsage Examples
Operator effectiveness analysis
The data below was calculated by running the tool on projects in Aptos' Move Framework.
The "Kill Rate" column shows the percentage of mutants killed by tests. While a high kill rate indicates good test coverage, operators with lower kill rates (more surviving mutants) are often more effective at detecting test gaps, as surviving mutants reveal untested code paths.