Skip to content

Commit

Permalink
Use ci-perf-kit 0.7.4: ignore runs with unmatched key/value in the lo…
Browse files Browse the repository at this point in the history
…gs. (#1025)

The perf CI run for mutator performance recently failed for the
following error:
```console
++ python3 scripts/mutator_report.py /home/gitlab-runner/actions-runner/_work/mmtk-core/mmtk-core/ci-perf-kit/result_repo/mutator /home/gitlab-runner/actions-runner/_work/mmtk-core/mmtk-core/reports/master
Traceback (most recent call last):
  File "/home/gitlab-runner/actions-runner/_work/mmtk-core/mmtk-core/ci-perf-kit/scripts/mutator_report.py", line 43, in <module>
    run_id, results = parse.parse_run(os.path.join(result_repo_mutator_root, l))
  File "/home/gitlab-runner/actions-runner/_work/mmtk-core/mmtk-core/ci-perf-kit/scripts/parse.py", line 82, in parse_run
    results.append(parse_log(os.path.join(log_folder, l), n_invocations))
  File "/home/gitlab-runner/actions-runner/_work/mmtk-core/mmtk-core/ci-perf-kit/scripts/parse.py", line 48, in parse_log
    assert len(mmtk_keys) == len(mmtk_values), "Error when reading MMTk statistics: num of keys does not match num of values"
AssertionError: Error when reading MMTk statistics: num of keys does not match num of values
```

One of the runs recently has an incomplete `MMTk Statistics` printing.
The process quit for some reason before it prints the values. This seems
to be a random issue, as the other runs in the same log file look fine.
```console
============================ MMTk Statistics Totals ============================
GC time.other time.stw time.jitc time.ygc collections.young time.ogc collections.old
```
The unmatched key/value pairs caused an assertion failure in the
`ci-perf-kit` script.`ci-perf-kit 0.7.4`
(https://github.com/mmtk/ci-perf-kit/releases/tag/0.7.4) fixes this.
Runs that do not have valid key/value pairs will be ignored, and the
script will not quit.
  • Loading branch information
qinsoon authored Nov 15, 2023
1 parent 2d5bae7 commit da9fb1f
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 8 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/micro-bm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
with:
repository: mmtk/ci-perf-kit
token: ${{ secrets.GITHUB_TOKEN }}
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
submodules: true
# Use rust-toolchain in the trunk (it doesnt matter much - if the toolchains defined in the trunk and the branch are different, we cant run anyway)
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/perf-baseline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
with:
token: ${{ secrets.CI_ACCESS_TOKEN }}
repository: mmtk/ci-perf-kit
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
submodules: true
# setup
Expand Down Expand Up @@ -94,7 +94,7 @@ jobs:
with:
token: ${{ secrets.CI_ACCESS_TOKEN }}
repository: mmtk/ci-perf-kit
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
submodules: true
# setup
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/perf-compare-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ jobs:
with:
repository: mmtk/ci-perf-kit
token: ${{ secrets.CI_ACCESS_TOKEN }}
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
submodules: true
# setup
Expand Down Expand Up @@ -219,7 +219,7 @@ jobs:
with:
repository: mmtk/ci-perf-kit
token: ${{ secrets.CI_ACCESS_TOKEN }}
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
submodules: true
# setup
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/perf-regression-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
uses: actions/checkout@v2
with:
repository: mmtk/ci-perf-kit
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
token: ${{ secrets.CI_ACCESS_TOKEN }}
submodules: true
Expand Down Expand Up @@ -116,7 +116,7 @@ jobs:
uses: actions/checkout@v2
with:
repository: mmtk/ci-perf-kit
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
token: ${{ secrets.CI_ACCESS_TOKEN }}
submodules: true
Expand Down Expand Up @@ -191,7 +191,7 @@ jobs:
uses: actions/checkout@v2
with:
repository: mmtk/ci-perf-kit
ref: "0.7.3"
ref: "0.7.4"
path: ci-perf-kit
token: ${{ secrets.CI_ACCESS_TOKEN }}
submodules: true
Expand Down

0 comments on commit da9fb1f

Please sign in to comment.