Test coverage

[wjakethompson]

JOSS review criterion: Automated tests (openjournals/joss-reviews#5779)

Using covr::package_coverage(), I see only 27.11% test coverage. I think this should be substantially higher in order to adequately verify that the package functions as expected and to ensure that code is immune to bugs that may be introduced by future changes.

covr::package_coverage()
konfound Coverage: 27.11%
R/cop_pse_auxiliary.R: 0.00%
R/helper_output_print.R: 0.00%
R/helper_output_table.R: 0.00%
R/helper_plot_correlation.R: 0.00%
R/helper_plot_threshold.R: 0.00%
R/konfound-glm.R: 0.00%
R/test_cop.R: 0.00%
R/test_pse.R: 0.00%
R/tkonfound_fig.R: 0.00%
R/test_sensitivity.R: 29.03%
R/pkonfound.R: 30.61%
R/nonlinear_auxiliary.R: 36.82%
R/tkonfound.R: 41.62%
R/test_sensitivity_ln.R: 43.81%
R/helper_output_dataframe.R: 44.44%
R/zzz.R: 50.00%
R/mkonfound.R: 60.78%
R/konfound.R: 66.13%
R/konfound-glm-dichotomous.R: 100.00%
R/konfound-lm.R: 100.00%
R/konfound-lmer.R: 100.00%

[fabian-s]

Seconded!

[jrosen48]

Hi both, as an update, we are now at 77%, and think we can get the coverage to above 90% with a little more digging. There are a few functions with extremely thorny calculations (nonlinear_auxiliary.R, particularly) that are rarely called.

[jrosen48]

We've added the package to codecov, so the test coverage can now be examined here: https://app.codecov.io/gh/konfound-project/konfound/tree/master/R

[jrosen48]

As an update, here's where the test coverage stands - all functions > 70% coverage and 80% total coverage across the package:

> covr::package_coverage()
konfound Coverage: 80.39%
R/nonlinear_auxiliary.R: 70.43%
R/test_sensitivity.R: 70.73%
R/test_sensitivity_ln.R: 73.81%
R/test_pse.R: 79.07%
R/tkonfound.R: 81.03%
R/pkonfound.R: 81.93%
R/tkonfound_fig.R: 83.02%
R/test_cop.R: 84.06%
R/core-sensitivity-mkonfound.R: 85.19%
R/helper_output_dataframe.R: 85.19%
R/cop_pse_auxiliary.R: 88.94%
R/konfound.R: 92.31%
R/helper_output_print.R: 95.60%
R/helper_plot_correlation.R: 95.83%
R/mkonfound.R: 95.83%
R/helper_plot_threshold.R: 97.73%
R/helper_output_table.R: 100.00%
R/konfound-glm-dichotomous.R: 100.00%
R/konfound-glm.R: 100.00%
R/konfound-lm.R: 100.00%
R/konfound-lmer.R: 100.00%
R/zzz.R: 100.00%

Noting that this is related to #50. We will use that issue primarily to discuss other code quality issues (cyclomatic complexity and non-compliant coding style).

[jrosen48]

We are at 81% and pausing due to asymptotic returns, in part due to the nature of some of the functions that calculate statistical indices and the error handling in their code. @wjakethompson and @fabian-s, can you offer any feedback on this? In your view, is this sufficient for ensuring the package functions as expected and that future bugs are likely to be detected? In our view, it is, as the coverage combined with the care we have taken to ensure that the statistical output is correct lend us confidence in the functioning of the package, though we recognize it could be possible to bolster further coverage (possibly to around 85-90% before severely diminishing returns set in!).

> covr::package_coverage()
konfound Coverage: 80.49%
R/test_sensitivity.R: 70.73%
R/nonlinear_auxiliary.R: 70.84%
R/test_sensitivity_ln.R: 74.00%
R/test_pse.R: 79.07%
R/tkonfound.R: 81.03%
R/pkonfound.R: 81.93%
R/tkonfound_fig.R: 83.02%
R/test_cop.R: 84.06%
R/core-sensitivity-mkonfound.R: 85.19%
R/helper_output_dataframe.R: 85.19%
R/cop_pse_auxiliary.R: 88.94%
R/konfound.R: 92.31%
R/helper_output_print.R: 95.60%
R/helper_plot_correlation.R: 95.83%
R/mkonfound.R: 95.83%
R/helper_plot_threshold.R: 97.73%
R/helper_output_table.R: 100.00%
R/konfound-glm-dichotomous.R: 100.00%
R/konfound-glm.R: 100.00%
R/konfound-lm.R: 100.00%
R/konfound-lmer.R: 100.00%
R/zzz.R: 100.00%

[fabian-s]

Seems ok, close if you want

[jrosen48]

thx