How about measuring test coverage to our code?

[ydah]

I felt from the reviews I have received here that it would be nice to be able to see test code coverage.
For example, if there are areas where there is currently not much test coverage, adding a test there would be a good thing for us.

[pirj]

With #1532 and #1533 as reference examples - I'm totally convinced.

[nijikon]

+1 on this on my end. Maybe as a start, we can add SimpleCov.minimum_coverage with the current coverage value so we at least won't get lower than what we have, and that will require people to cover for new cops. We can add more coverage and bump the value as we develop.

[ydah]

As a first small step, I've opened the following I'd like to hear what you think.

• Add simplecov to measure test coverage of our code #1534

[bquorning]

SimpleCov was removed from the repo about a year ago, in #1084, mostly because we didn’t really use Code Climate anymore. 😅

On other projects, I have sometimes struggled with a “100% test coverage” requirement, but that was especially around certain code paths running exclusively for e.g. certain versions of Ruby, or ActiveRecord etc. – which means that in one test run, we would never reach 100%.

Perhaps that is not a problem for us? I know that with SimpleCov it’s possible to merge coverage results from different test runs, so that might be the solution if the problem should occur.

[pirj]

Fair enough. However, our cops don't seem to care about Ruby or RSpec versions.
At some point we may decide to target RSpec 3 and RSpec 4 differently with some minimum_target_rspec_version similarly to how rubocop and rubocop-rails do. But this will be done in the same test run, so the coverage will be complete.

[ydah]

Oh. It was deleted once, wasn't it? I did not know that. 😅

In testing RuboCop, as @pirj mentioned, I think we are doing all the testing in a one-shot RSpec run, since we can switch by parameters as follows.

https://github.com/rubocop/rubocop/blob/aaa4166b1f8fadcd136537c3a2b7ad13bb1de907/spec/rubocop/cop/lint/unified_integer_spec.rb#L39

context 'target ruby version >= 2.4', :ruby24 do
  # ...
end

[pirj]

That's a good observation, I completely missed that we're actually using :ruby27 metadata in quite some places.
Under the hood, it sets the TargetRubyVersion that affects if a spec will be run on a certain Ruby version or not. Those with :ruby27 won't e.g. run on Ruby 2.6.
But it's a specification for a minimum required Ruby version, and there seem to be no maximum one yet.

If we run our CI jobs on the latest Ruby, 3.2 presently, all those specs would still run.