Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different coverage results for scala 2 and scala 3 #21877

Open
goshacodes opened this issue Nov 2, 2024 · 3 comments
Open

Different coverage results for scala 2 and scala 3 #21877

goshacodes opened this issue Nov 2, 2024 · 3 comments
Assignees
Labels
area:coverage Code coverage, see https://dotty.epfl.ch/docs/internals/coverage.html itype:bug

Comments

@goshacodes
Copy link

goshacodes commented Nov 2, 2024

Compiler version

3.3.4

Problem

Scala 3 coverage of my project is still significantly lower than scala 2 (~ 20% for statements and 27% for branches)
I've tried to inspect reports to see differences.

  1. In general Scala 3 has much more statements and branches. Why this may be so?
  2. Scala 3 has much more classes then Scala 2. And looks like this is because of $anon class. I've tried to look closer and now macro generated code is also covered. IMO macro generated by some library code should not be covered by default, since library users are not testing libraries, but their own code.

So is it just more precise score and this is a desired behavior or not?

Scala 2.13 report

image

Scala 3 report

image

Scala 2.13 report

image image

Scala 3 report

image image

Statements

image image
@goshacodes goshacodes added the stat:needs triage Every issue needs to have an "area" and "itype" label label Nov 2, 2024
@Gedochao Gedochao added itype:question area:coverage Code coverage, see https://dotty.epfl.ch/docs/internals/coverage.html and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Nov 4, 2024
@Gedochao
Copy link
Contributor

Gedochao commented Nov 4, 2024

cc @KacperFKorban @sjrd

@KacperFKorban
Copy link
Member

@goshacodes Thanks for the issue. It is probably a bug, Scala 3's coverage generation mostly just tries to be as close to Scala 2's coverage results as possible.
Can you provide some reproduction code/steps for generating your coverage data? A minimized snippet is ideal, but for coverage a repo will also do.

@goshacodes
Copy link
Author

goshacodes commented Nov 5, 2024

@KacperFKorban prepared an example

https://github.com/goshacodes/coverage-example

Reports are in the coverage-reports directory

To reproduce you need something like this:
sbt ";++2.13; coverage; test; coverageReport; ++3.3.4; test; coverageReport"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:coverage Code coverage, see https://dotty.epfl.ch/docs/internals/coverage.html itype:bug
Projects
None yet
Development

No branches or pull requests

3 participants