_r_val_hub_repos_updates.qmd

---
---

# 📦 Repositories

## Repositories Workstream

::: { style="font-size: 1.2em;" }
> Supporting a transparent, open, dynamic, cross-industry approach of
> establishing and maintaining a _repository_ of R packages.
:::

* A **CRAN**-like repository
* Providing quantifiable package qualities for risk-based decision-making
* Evaluated against representative systems
* "Bring-your-own" quality cut-offs
* Declarative quality decision-making

## The Pulse of the Industry

* Our whitepaper is widely adopted
* But implementing it is inconsistent & laborious
  * Variations throughout industry pose uncertainty
  * Sharing software with health authorities is a challenge
  * Health authorities, overwhelmed by technical inconsistencies, are more
    likely to question software use
* We feel the most productive path forward is a shared ecosystem
* Public discussion on how to characterize quality code/methods

## Goals

::: { .columns }
::: { .column .panel style="width: 40%; font-size: 0.95em;" }
### Generating Quality Indicators

* Provide a community-maintained catalog of package quality indicators ("risk
  metrics")
* Calculated against cohort of packages
* Known system
* Consistently evaluated, with transparent methods
:::

::: { .column .panel style="width: 40%; font-size: 0.95em;" }
### Consolidate Decision-Making

* Serve subsets of packages that conform to a specified risk tolerance
* Transparently demonstrate selection criteria
* Allows for one-off-analysis from public repo
* .. or mirroring of filtered snapshot
:::
:::

## An evolving R ecosystem

> In close communication with many beloved R projects

:::{ .columns .font-size-small }
:::{ .column .center .panel width="25%" }
![](resources/rc-logo.png)

### Submissions Working Group
:::

:::{ .column .center .panel width="25%" }
![](resources/rc-logo.png)

### Repositories Working Group
:::

:::{ .column .center .panel width="25%" }
![](resources/pharmaverse.png){ width="30%" }

### `pharmaverse`
:::

:::{ .column .center .panel width="25%" }
![](resources/r-hub.png){ width="50%" }

targetting `repos` integration
:::

:::{ .column .center .panel width="25%" }
![](resources/r-lib.png){ width="40%" }

### `r-lib/pak`

targetting `pak` integration
:::
:::

## Pilot Implementation

_focus on proving capabilities, quick development_

![](resources/pilot.drawio.svg){ style='width: 100%; height: 100%; margin: 2em 0;' }

✨ _all modelled after [r-hub/repos](https://github.com/r-hub/repos)_

# Interacting with the repo

## Packages risk filters

::: { .columns }
::: {.column width="50%"}
- Helper package for system administrators
- Restricts packages available for installation to those fitting a policy
- Uses packages metadata in the repo
- May be used together with manual checks (e.g., read a statistical review)
:::

::: {.column width="50%"}
![](resources/package-filters.drawio.svg){ width="100%" height="13em" }
:::
:::

## As a user*

::: { style="font-size: 1.35em;" }
```r
repo <- "https://raw.githubusercontent.com/pharmaR/repos/main/ubuntu-22.04/4.5"
options(repos = c("pharmaR/repos/ubuntu" = repo))
```
:::

Define your quality expectations _in code_

::: { style="font-size: 1.35em;" }
```{.r code-line-numbers="2-3,5-8,10-12,14-17"}
options(available_packages_filters = risk_filter(
  # no known security vulnerabilities
  quality_cve_count == 0 & (

    # package is exceptionally testing 
    (quality_code_coverage >= 0.8 & 
      quality_example_coverage >= 0.8 &
      quality_r_cmd_check_errors == 0) |

    # or is exceptionally well adopted
    dplyr::percent_rank(quality_downloads_1yr) > 0.9 |
    quality_reverse_dependencies_count >= 10 |

    # or seems to follow thorough development practices
    (quality_has_website &
      quality_vignette_count >= 1 &
      quality_author_count >= 3)
  )
))
```

<small>\*aspirational deviations from proof of concept in [github.com/pharmaR/pharmapkgs]()</small>
:::

## As a user*

::: { style="font-size: 1.35em;" }
```r
repo <- "https://raw.githubusercontent.com/pharmaR/repos/main/ubuntu-22.04/4.5"
options(repos = c("pharmaR/repos/ubuntu" = repo))
```
:::

Make exceptions _explicit_

::: { style="font-size: 1.35em;" }
```{.r code-line-numbers="2"}
options(available_packages_filters = risk_filter(
  exceptions = c("riskmetric", "riskscore"),

  # no known security vulnerabilities
  quality_cve_count == 0 & (

    # package is exceptionally testing 
    (quality_code_coverage >= 0.8 & 
      quality_example_coverage >= 0.8 &
      quality_r_cmd_check_errors == 0) |

    # or is exceptionally well adopted
    dplyr::percent_rank(quality_downloads_1yr) > 0.9 |
    quality_reverse_dependencies_count >= 10 |

    # or seems to follow thorough development practices
    (quality_has_website &
      quality_vignette_count >= 1 &
      quality_author_count >= 3)
  )
))
```

<small>\*aspirational deviations from proof of concept in [github.com/pharmaR/pharmapkgs]()</small>
:::

## As a user*

::: { style="font-size: 1.35em;" }
```r
repo <- "https://raw.githubusercontent.com/pharmaR/repos/main/ubuntu-22.04/4.5"
options(repos = c("pharmaR/repos/ubuntu" = repo))
```
:::

Tools for inspecting packages that do not adhere to pre-specified criteria

::: { style="font-size: 1.35em;" }
```{.r}
risk_explain(package = "options")
#> <criteria>
#> * `quality_code_coverage` of `0.43` (< 0.8)
#> * `dplyr::precent_risk(quality_downloads_1yr)` of `0.2` (< 0.9)
#> * `quality_reverse_dependencies_count` of `0` (< 10)
#> * `quality_author_count` of `1` (< 3)
```

<small>\*aspirational deviations from proof of concept in [github.com/pharmaR/pharmapkgs]()</small>
:::

## As an Administrator

::: { style="font-size: 1.35em;" }
```r
repo <- "https://raw.githubusercontent.com/pharmaR/repos/main/ubuntu-22.04/4.5"
options(repos = c("pharmaR/repos/ubuntu" = repo))
```
:::

Applying filters to a repository mirror 

::: { style="font-size: 1.35em;" }
```{.r}
options(available_packages_filters = risk_filter(
  # define your criterion in code
))

packages <- available.packages()
write.dcf(packages, "FILTERED_PACKAGES")
# sync repo using PACKAGES file
```
:::

# How it Works

## Shamelessly Piggybacking on `r-hub/repos`

::: columns
::: { .incremental .column width="50%" }
- `github.com/cran` mirrors CRAN packages, hosting source code & binary builds
- `r-hub/repos` indexes packages mirrored on `github.com/cran`
  - Adds a custom `DownloadUrl` field, used by `{pak}` to fetch packages
  - Serves `PACKAGES` file, a standard format for indexing packages
- `pharmaR/repos` periodically looks for changes in `r-hub/repos`
  - Calculates metrics for updated packages
  - And hosts `PACKAGES` file with additional quality fields
:::

::: {.column width="50%"}

`r-hub/repos` `PACKAGES` file

```{.yaml code-line-numbers="5-6"}
Package: bslib
Version: 0.6.1
Depends: R (>= 2.10), R (>= 4.4), R (< 4.4.99)
License: MIT + file LICENSE
DownloadURL:
  https://github.com/cran/bslib/../bslib-ubuntu-22.04.tar.gz
...
```

Added fields for risk-based assessment

```{.yaml}
riskmetric_run_date: 2023-06-21
riskmetric_version: 0.2.1
quality_code_coverage: 0.852
quality_vignettes_count: 1
quality_r_cmd_check_errors: 0
...
```
:::
:::

::: {.notes}
Yann
:::

## Package Validation Workflow

### Risk assessment pipeline

:::: {.columns}
::: { .column .center width="32%" }
<div style="filter: grayscale() brightness(150%) contrast(30%) sepia() saturate(300%) hue-rotate(2.9rad);">
![](resources/package-inspect-svgrepo-com.svg){width="50%" height="50%"}
</div>

Calculates package quality metadata for updated packages and reverse dependencies
:::

::: {.column .center width="32%"}
<div style="filter: contrast(10%) sepia() saturate(1000%) hue-rotate(3.3rad);">
![](resources/document-code-2-svgrepo-com.svg){ width="50%" height="50%" }
</div>

Produces logs and necessary reporting metadata
:::

::: {.column .center width="32%"}
<div style="filter: contrast(10%) sepia() saturate(1000%) hue-rotate(3.3rad);">
![](resources/house-svgrepo-com.svg){ width="50%" height="50%" }
</div>

Support in-house packages with open-source tooling
:::
::::

# Our roadmap

## What's next

_Automating up-to-date quality metrics to support sponsor risk assessment_

![](resources/next-steps.drawio.svg){ style='width: 100%; height: 100%; margin: 2em 0;' }

## What's next
### Technical Development

:::: { .columns }
::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
🧪
:::

### Pipeline Development

_in progress_

metric re-calculation
:::

::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
📜
:::

### Artifact Discoverability

Accessible logs, build artifacts
:::

::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
🥳
:::
### Friendly User Interfaces

Comfy tools for working with the repo
:::
::::

## What's next
### Governance

:::: { .columns }
::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
🚢
:::

### Pipeline Development

_in progress_

Base image development/exploration
:::

::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
🏅
:::

### Filter Recommendations

Start dialogue about industry-standard thresholds
:::

::: { .column .panel .center style="height: 70%; width: 29%;" }
::: { .blueify .font-size-huge }
♻️
:::
### Lifecycle Decisions

- how frequently should we sync with `r-hub/repos`/`CRAN`?
- which packages require re-evaluation?
:::
::::