Metric: number of examples

[llrs-roche]

Description
Documentation pages with runnable(?) examples

Varies based on ...

• Take into account: \donttest{}, \dontrun{} ?

Sometimes there is interest on a ratio, the ratio can change depending on which other metric is used:

• Metric: Number of exported objects (this is not as simple as I expected)
• Metric: Number of help pages (There might be some internal help pages)

[dominik-appsilon]

I started work on metric-documentation-examples branch. It is already functional.

Metrics

The branch contains three new metrics for assessing the quality and completeness of package documentation:

help_pages_with_examples_count (integer)

Absolute count of help pages that include an \examples section.

Rationale: Provides a simple, concrete measure of how much example code exists in the documentation. Higher counts generally indicate more effort put into demonstrating package usage. {dplyr} has 80 help pages with examples out of 115 total help pages.

help_examples_coverage (numeric, 0.0-1.0)

Fraction of help pages that include \examples sections, calculated as help_pages_with_examples_count / help_page_count.

Rationale: Normalizes the example count by the total number of help pages, making it comparable across packages of different sizes. A value of 0.696 (as in {dplyr}) meeans 69.6% of help pages include examples.

exports_help_coverage (numeric, 0.0-1.0)

Fraction of exported objects that have associated help pages, calculated by matching package exports (from NAMESPACE) against \alias tags in help pages.

Rationale: Identifies undocumented exports, which are a significant documentation gap. A value of 1.0 means all exports are documented; lower values indicate missing documentation.

Implementation

The implementation maps exports to help pages using \alias tags from Rd files. The idea is:

1. Parse NAMESPACE using parseNamespaceFile() to get all exports
2. Load help database using tools::Rd_db()
3. Extract \alias tags from each help page to build export-to-page mapping
4. Check which help pages contain \examples sections
5. Calculate coverage metrics

This process generates a few items of auxiliary data:

• exported_count: Total exports
• help_page_count: Total help pages
• documented_exports_count: Exports with help pages
• undocumented_exports: Character vector of export names lacking documentation
• help_pages_with_examples: Character vector of help page names containing examples

Honestly, which data got exposed as "metric" was partially based on my "gut feeling" what will be useful. I am open for suggestions how to do it differently.

Discussion

I opted not to distinguish \dontrun and \donttest tags as their extraction can be tricky and the way they are used can be ambiguous. For example counting "runnable vs. non-runnable" examples would penalize packages that thoughtfully include examples of error handling, invalid usage patterns, or resource-intensive operations. Again, I am open for discussing this topic.