Analysing a release

The code is currently in lib/release_analyser.rb

A release is delimited by the head_sha (the commit at the HEAD of the branch at the time of deploy), and the "starting" SHA, which is the previous release's HEAD commit.

We need at least these two delimiting commits in order to analyse a release.

Analysing the release means identifying which pull requests contributed to it, and then collecting all the stats we want to store in InfluxDB for each PR, such as seconds_since_first_commit, total_line_changes_in_pr, etc.

Each data point is tagged with the project, the environment, the pull request number, and the deploy SHA, to facilitate grouping and filtering the data.

The fields

Generic fields

• The timestamp, _time, is what Influx will interpret as the time the "measured event" happened, and it must be a Unix timestamp. It is inserted as time and referred to within Influx as _time.

Each data point needs a timestamp, otherwise Influx will use the time of importing the data, which is not what we want.

We are using the time the deployment has finished (as tracked by GitHub Actions workflows for the historical script, and as reported by the health check endpoint for the latest release script).

• The special measurement field (referred to as _measurement within Influx) corresponds to the "name", and it is how we can filter for the entire stream of these specific "measurements". (There can be any number of measurements in a bucket, but we have chosen to only use one type of measurement.)

• The tags project, env, and deploy_sha will allow us to filter and group projects, environments, and individual deploys.

Pull request tags and fields

Pull request data is meant to serve our goals of measuring how long it takes our work to go from start to being available to users on production, and identifying where there are any slow points in the development pipeline.

The tag pr allows grouping and filtering for a specific PR.

Time-related measurements

• the oldest commit in the PR is a proxy for when work on a specific story started
• when PRs are opened / merged, to measure whether there are any bottlenecks in our review processes
• the time when the work was deployed

Code-related measurements

• total number of commits in a PR
• total line changes in a PR
• average number of line changes per commit in a PR
• number of reviews on a PR
• number of comments on a PR