Add code-coverage-tools package to compare code coverage on the PR build. #22452
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
area: build
added 2 commits
September 10, 2024 12:54
github-actions
bot
removed
the
dependencies
jatgarg
changed the title
tylerbutler
reviewed
Sep 27, 2024
build-tools/packages/build-cli/src/commands/report/codeCoverageStats.ts
Outdated
Show resolved
Hide resolved
build-tools/packages/build-cli/src/commands/report/codeCoverageStats.ts
Outdated
Show resolved
Hide resolved
build-tools/packages/build-cli/src/commands/report/codeCoverageStats.ts
Outdated
Show resolved
Hide resolved
build-tools/packages/build-cli/src/commands/report/codeCoverageStats.ts
Outdated
Show resolved
Hide resolved
build-tools/packages/build-cli/src/commands/report/codeCoverageStats.ts
Outdated
Show resolved
Hide resolved
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
…eStats.ts Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
…to coveragecmnd
|
Nit: it would probably be good to call out in the PR description why we needed to create our own coverage infrastructure, rather than using an off-the-shelf solution. |
tylerbutler
approved these changes
Oct 3, 2024
build-tools/packages/build-cli/src/codeCoverage/getCoverageMetrics.ts
Outdated
Show resolved
Hide resolved
build-tools/packages/build-cli/src/codeCoverage/getCoverageMetrics.ts
Outdated
Show resolved
Hide resolved
| logger?.verbose(`Found baseline build with id: ${baselineBuild.id}`); | ||
|
|
||
| return { | ||
| build: { ...baselineBuild, id: baselineBuild.id }, |
There was a problem hiding this comment.
not a blocker, but I am still a bit confused here. if baselineBuild.id === undefined, then you throw earlier, so baselineBuild.id must be a defined value by the time you get to this line. So what is this code changing in the data? It seems like it's just setting the id property to itself, which should be a no-op.
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Co-authored-by: Tyler Butler <tyler@tylerbutler.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
AB#14170
Add code-coverage-tools package to compare code coverage on the PR build.
1.) Before running coverage comparison, code coverage plugin identifies the baseline build for the PR.
2.) Once the baseline build is identified, we download the build artifacts corresponding to the
Code Coverage Report_<Build_Number>artifact name for this build3.) We then collect the code coverage stats for the PR build and then make the comparison with the baseline.
4.) If the code coverage diff (branch coverage) is more than a percentage point change, then we fail the build for the PR. We also fail the build in case the code coverage for the newly added package is less than 50%.
5.) We post the comment on the PR specifying the code coverage change if any for each package which is modified.
We needed this separate module as we need specifics to do the code coverage comparison like the baseline with which we need to make the comparison and then what we need to compare and then after comparison what comment and in what format we want to report it.
Sample Comment: