feature/coverage-capture-and-merge

[bwalsh]

Feature Request: End-to-End Coverage Capture and Merge Workflow

--
 

Use Case

Teams want a repeatable way to capture coverage from integration/end-to-end runs (that execute the CLI binaries) and merge that data with unit test coverage. Go 1.20+ provides GOCOVERDIR support for instrumented binaries, but the current workflow is manual and error-prone. This feature request proposes a documented, script-driven flow that:
 

• Builds an instrumented git-drs binary for integration tests.
• Executes the existing end-to-end script while writing raw coverage data.
• Converts raw coverage data into a standard coverage profile.
• Merges unit test coverage data with integration coverage data into a combined profile.

See #130  

Integrated into the workflow:

• developer makes change
• developer runs integration tests
• developer commits integration-coverage-file
• developer pushes
• github action runs unit tests
• a new unit test checks integration-coverage-file, compares to most recent *.go timestamp, fails if <
• github action merges coverage files
• github action updates coverage %

Proposed Solution

Add and document two scripts:
 

1. tests/scripts/coverage/run-e2e-coverage.sh

• Builds an instrumented git-drs binary (go build -cover -covermode=atomic -coverpkg=./...).
• Sets GOCOVERDIR to capture raw coverage data.
• Runs the existing end-to-end test script.
• Writes a profile using go tool covdata textfmt.
   

2. tests/scripts/coverage/combine-coverage.sh

• Merges raw unit-test and integration coverage directories via go tool covdata merge.
• Emits a combined profile via go tool covdata textfmt.
   

Acceptance Tests

1. Integration coverage capture

• Given the e2e prerequisites are configured, running:

tests/scripts/coverage/run-e2e-coverage.sh

creates a coverage profile at coverage/integration/coverage.out.
 
2. Coverage merge

• Given unit tests are run with GOCOVERDIR=coverage/unit/raw and integration coverage is captured into coverage/integration/raw, running:

tests/scripts/coverage/combine-coverage.sh

creates coverage/combined.out.
 
3. Combined profile is readable

• Running:

go tool cover -func=coverage/combined.out

prints coverage data without errors.
 

Shell Scripts

 

tests/scripts/coverage/run-e2e-coverage.sh

#!/usr/bin/env bash
set -euo pipefail
 
ROOT_DIR=$(git rev-parse --show-toplevel)
COVERAGE_ROOT="${COVERAGE_ROOT:-${ROOT_DIR}/coverage}"
INTEGRATION_COV_DIR="${INTEGRATION_COV_DIR:-${COVERAGE_ROOT}/integration/raw}"
INTEGRATION_PROFILE="${INTEGRATION_PROFILE:-${COVERAGE_ROOT}/integration/coverage.out}"
BUILD_DIR="${BUILD_DIR:-${ROOT_DIR}/build/coverage}"
 
E2E_SCRIPT="${ROOT_DIR}/tests/scripts/end-2-end/e2e.sh"
if [[ ! -x "${E2E_SCRIPT}" ]]; then
ALT_E2E_SCRIPT="${ROOT_DIR}/tests/scripts/end-to-end/e2e.sh"
if [[ -x "${ALT_E2E_SCRIPT}" ]]; then
E2E_SCRIPT="${ALT_E2E_SCRIPT}"
else
echo "Unable to find executable e2e.sh at ${E2E_SCRIPT} or ${ALT_E2E_SCRIPT}." >&2
exit 1
fi
fi
 
mkdir -p "${BUILD_DIR}" "${INTEGRATION_COV_DIR}" "$(dirname "${INTEGRATION_PROFILE}")"
 
GOFLAGS=()
if [[ -n "${GOFLAGS_EXTRA:-}" ]]; then
GOFLAGS+=("${GOFLAGS_EXTRA}")
fi
 
go build "${GOFLAGS[@]}" -cover -covermode=atomic -coverpkg=./... -o "${BUILD_DIR}/git-drs" .
 
export PATH="${BUILD_DIR}:${PATH}"
export GOCOVERDIR="${INTEGRATION_COV_DIR}"
 
"${E2E_SCRIPT}"
 
go tool covdata textfmt -i="${INTEGRATION_COV_DIR}" -o "${INTEGRATION_PROFILE}"
 
echo "Integration coverage profile saved to ${INTEGRATION_PROFILE}"

 

tests/scripts/coverage/combine-coverage.sh

#!/usr/bin/env bash
set -euo pipefail
 
ROOT_DIR=$(git rev-parse --show-toplevel)
COVERAGE_ROOT="${COVERAGE_ROOT:-${ROOT_DIR}/coverage}"
 
UNIT_COV_DIR="${1:-${UNIT_COV_DIR:-${COVERAGE_ROOT}/unit/raw}}"
INTEGRATION_COV_DIR="${2:-${INTEGRATION_COV_DIR:-${COVERAGE_ROOT}/integration/raw}}"
MERGED_COV_DIR="${3:-${MERGED_COV_DIR:-${COVERAGE_ROOT}/merged/raw}}"
COMBINED_PROFILE="${4:-${COMBINED_PROFILE:-${COVERAGE_ROOT}/combined.out}}"
 
if [[ ! -d "${UNIT_COV_DIR}" ]]; then
echo "Unit coverage directory not found: ${UNIT_COV_DIR}" >&2
echo "Run unit tests with GOCOVERDIR set, e.g.:" >&2
echo "  GOCOVERDIR=${COVERAGE_ROOT}/unit/raw go test -cover ./..." >&2
exit 1
fi
 
if [[ ! -d "${INTEGRATION_COV_DIR}" ]]; then
echo "Integration coverage directory not found: ${INTEGRATION_COV_DIR}" >&2
exit 1
fi
 
mkdir -p "${MERGED_COV_DIR}" "$(dirname "${COMBINED_PROFILE}")"
 
rm -rf "${MERGED_COV_DIR:?}"/*
 
go tool covdata merge -i="${UNIT_COV_DIR},${INTEGRATION_COV_DIR}" -o "${MERGED_COV_DIR}"
go tool covdata textfmt -i="${MERGED_COV_DIR}" -o "${COMBINED_PROFILE}"
 
echo "Combined coverage profile saved to ${COMBINED_PROFILE}"