Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

142 changes: 142 additions & 0 deletions tools/token-metrics/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Spectrum Token Metrics

Extracts and reports design token metrics from `@adobe/spectrum-tokens` for tracking the health, coverage, and growth of the Spectrum design token system.

This tool supports the OKR: **"Token metrics are defined and included into our metrics system & have a baseline to compare moving forward."**

## What This Answers for Leadership

**"What does 'token metrics' mean specifically, and when should this be done by?"**

Token metrics are quantitative measurements extracted from the design token data in `packages/tokens` that give us visibility into the health, maturity, and coverage of Spectrum's design data layer. They establish a baseline so we can measure progress over time and make data-informed decisions about where to invest in the token system.

## Metrics Available Today (Baseline)

These metrics are extracted directly from the current token source files with zero additional instrumentation:

### 1. Token Inventory (Health)

| Metric | Current Value | Why It Matters |
| ---------------------------------- | ------------------------ | ------------------------------------------------------------------- |
| **Total tokens** | 2,338 | Size of the design data surface area |
| **Active tokens** | 2,150 | Tokens currently in use |
| **Deprecated tokens** | 188 (8.0%) | Technical debt in the token layer |
| **Deprecated with migration path** | 57 (30.3% of deprecated) | How well we support migration when tokens change |
| **Private tokens** | 372 | Internal implementation tokens not intended for direct consumer use |

### 2. Token Scope (Architecture)

| Metric | Current Value | Why It Matters |
| --------------------------------- | ------------- | ------------------------------------------------ |
| **Global (system) tokens** | 965 | Foundation tokens available to all components |
| **Component tokens** | 1,373 | Tokens scoped to specific components |
| **Unique components with tokens** | 95 | How many components have dedicated token support |

### 3. Token Architecture (Complexity)

| Metric | Current Value | Why It Matters |
| ---------------------------- | ------------- | -------------------------------------------------- |
| **Alias (reference) tokens** | 654 | Tokens that reference other tokens for consistency |
| **Direct-value tokens** | 459 | Tokens with hard-coded values |
| **Color theme set tokens** | 470 | Tokens with light/dark/wireframe variants |
| **Scale set tokens** | 755 | Tokens with desktop/mobile variants |
| **Max alias chain depth** | 4 | Deepest nesting of token references |

### 4. Data Quality

| Metric | Current Value | Why It Matters |
| --------------------------- | -------------------------- | ------------------------------------------------ |
| **UUID coverage** | 100% | Every token has a unique identifier for tracking |
| **Migration path coverage** | 30.3% of deprecated tokens | Gap in providing clear upgrade guidance |

### 5. Component Coverage

| Metric | Current Value | Why It Matters |
| ------------------------- | ------------- | ---------------------------------------------- |
| **Registered components** | 54 | Components in the design system registry |
| **With tokens** | 44 (81.5%) | Components that have design token support |
| **With schema** | 54 (100%) | Components that have formal schema definitions |

### 6. Semantic Categories

| Category | Count | What It Covers |
| ---------- | ----- | ------------------------------------------------- |
| Layout | 741 | Spacing, sizing, dimensions, margins |
| Other | 550 | Component-specific tokens not in other categories |
| Content | 289 | Text and content-related colors/styles |
| Typography | 284 | Font families, sizes, weights, line heights |
| Color | 227 | Color values and visual tokens |
| Background | 151 | Background colors and opacity |
| Border | 58 | Border colors and styles |
| Shadow | 23 | Drop shadows and elevation |
| Opacity | 9 | Opacity values |
| Icon | 6 | Icon-specific tokens |

## Additional Metrics Worth Working On

These require additional instrumentation or cross-system integration:

### Phase 2 (Recommended for Q2)

1. **Token adoption rate** - Track which tokens are actually used in downstream implementations (Spectrum CSS, React Spectrum, S2) via automated source code scanning.

2. **Token change velocity** - Tokens added/modified/deleted per release. The existing `release-analyzer` tool partially supports this; it can be extended to track token-level changes per version.

3. **Deprecation lifecycle duration** - How long deprecated tokens remain before removal. This measures how effectively we communicate and complete migrations.

4. **Cross-platform token parity** - Compare tokens available for web vs iOS vs Android to measure multi-platform readiness. This directly supports the "multi-platform philosophy" OKR item.

### Phase 3 (Q3+)

5. **Consumer migration completeness** - For each deprecated token, what percentage of known consumers have migrated. Requires integration with downstream repos.

6. **Token request/gap tracking** - Track feature requests for new tokens and measure time-to-delivery.

7. **Design-to-code token fidelity** - Measure how well design tool tokens map to implementation tokens (requires Figma API integration).

8. **Custom token prevalence** - Track how often teams create custom tokens outside the system, indicating gaps in the core offering.

## Usage

### Generate a full metrics report

```bash
node src/cli.js
```

### Print summary only

```bash
node src/cli.js --summary
```

### Output to a specific file

```bash
node src/cli.js --output metrics-2026-Q1.json
```

### Using moon

```bash
moon run token-metrics:metrics
```

## Integration with Metrics Systems

The tool outputs a JSON report that can be:

* Stored in the repo as a baseline snapshot (commit the output file)
* Ingested by dashboarding tools (Grafana, Datadog, etc.)
* Compared across releases to show trends
* Included in CI to track regressions (e.g., deprecation rate exceeding threshold)

## Suggested Timeline

| Milestone | Target | Description |
| --------------------- | ----------------- | --------------------------------------------- |
| Baseline established | **Now (Q1 2026)** | This tool generates the initial baseline |
| CI integration | Q1 2026 | Run metrics on every PR to detect regressions |
| Dashboard | Q2 2026 | Visualize metrics trends over time |
| Adoption metrics | Q2 2026 | Cross-reference with downstream consumers |
| Multi-platform parity | Q2-Q3 2026 | Compare web/iOS/Android token coverage |
7 changes: 7 additions & 0 deletions tools/token-metrics/ava.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export default {
files: ["test/**/*.test.js"],
verbose: true,
environmentVariables: {
NODE_ENV: "test",
},
};
7 changes: 7 additions & 0 deletions tools/token-metrics/moon.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
tasks:
test:
command: pnpm ava test
platform: node
metrics:
command: node src/cli.js
platform: node
43 changes: 43 additions & 0 deletions tools/token-metrics/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"name": "@adobe/spectrum-tokens-metrics",
"private": true,
"version": "0.1.0",
"description": "Extracts and reports design token metrics from @adobe/spectrum-tokens for tracking health, coverage, and growth of the Spectrum design token system",
"type": "module",
"main": "./src/index.js",
"bin": {
"token-metrics": "./src/cli.js"
},
"scripts": {
"test": "ava"
},
"engines": {
"node": ">=20.12.0",
"pnpm": ">=10.17.1"
},
"packageManager": "pnpm@10.17.1",
"repository": {
"type": "git",
"url": "git+https://github.com/adobe/spectrum-design-data.git",
"directory": "tools/token-metrics"
},
"author": "Garth Braithwaite <garthdb@gmail.com> (http://garthdb.com/)",
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/adobe/spectrum-design-data/issues"
},
"homepage": "https://github.com/adobe/spectrum-design-data/tree/main/tools/token-metrics#readme",
"keywords": [
"spectrum",
"tokens",
"metrics",
"design-system",
"analysis"
],
"dependencies": {
"commander": "^13.1.0"
},
"devDependencies": {
"ava": "^6.0.0"
}
}
130 changes: 130 additions & 0 deletions tools/token-metrics/src/cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
#!/usr/bin/env node

/*
Copyright 2024 Adobe. All rights reserved.
This file is licensed to you under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. You may obtain a copy
of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under
the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
OF ANY KIND, either express or implied. See the License for the specific language
governing permissions and limitations under the License.
*/

import { Command } from "commander";
import { writeFile } from "fs/promises";
import { resolve } from "path";
import generateMetricsReport from "./index.js";

const program = new Command();

program
.name("token-metrics")
.description(
"Extract and report design token metrics from @adobe/spectrum-tokens",
)
.version("0.1.0")
.option(
"-o, --output <path>",
"Output file path for JSON report",
"token-metrics-report.json",
)
.option("--tokens-src <path>", "Path to tokens/src directory")
.option("--registry <path>", "Path to design-system-registry components.json")
.option("--schemas <path>", "Path to component-schemas/schemas/components")
.option("--summary", "Print only the summary to stdout", false)
.action(async (options) => {
try {
const reportOptions = {};
if (options.tokensSrc)
reportOptions.tokensSrc = resolve(options.tokensSrc);
if (options.registry)
reportOptions.registryPath = resolve(options.registry);
if (options.schemas) reportOptions.schemasDir = resolve(options.schemas);

console.log("Generating token metrics report...\n");
const report = await generateMetricsReport(reportOptions);

if (options.summary) {
printSummary(report);
} else {
printSummary(report);
const outputPath = resolve(options.output);
await writeFile(outputPath, JSON.stringify(report, null, 2));
console.log(`\nFull report written to: ${outputPath}`);
}
} catch (error) {
console.error("Error generating metrics:", error.message);
process.exit(1);
}
});

function printSummary(report) {
const { summary, aliasAnalysis, uuidCoverage, componentCoverage } = report;

console.log("=== Spectrum Design Token Metrics ===\n");
console.log(`Generated: ${report.generatedAt}\n`);

console.log("--- Token Inventory ---");
console.log(` Total tokens: ${summary.totalTokens}`);
console.log(` Active tokens: ${summary.activeTokens}`);
console.log(
` Deprecated tokens: ${summary.deprecatedTokens} (${summary.deprecationRate}%)`,
);
console.log(
` Deprecated w/ migration: ${summary.deprecatedWithMigrationPath} (${summary.migrationPathCoverage}% coverage)`,
);
console.log(` Private tokens: ${summary.privateTokens}`);

console.log("\n--- Token Scope ---");
console.log(` Global tokens: ${summary.globalTokenCount}`);
console.log(` Component tokens: ${summary.componentTokenCount}`);
console.log(` Unique components: ${summary.uniqueComponents}`);

console.log("\n--- Token Architecture ---");
console.log(` Alias (reference) tokens: ${aliasAnalysis.aliasTokens}`);
console.log(
` Direct-value tokens: ${aliasAnalysis.directValueTokens}`,
);
console.log(
` Set-based tokens: ${aliasAnalysis.setBasedTokens.total}`,
);
console.log(
` Color theme sets: ${aliasAnalysis.setBasedTokens.colorTheme}`,
);
console.log(
` Scale sets: ${aliasAnalysis.setBasedTokens.scale}`,
);

console.log("\n--- Data Quality ---");
console.log(
` UUID coverage: ${uuidCoverage.coveragePercent}% (${uuidCoverage.withUuid}/${uuidCoverage.withUuid + uuidCoverage.withoutUuid})`,
);
console.log(
` Max alias chain depth: ${report.aliasChainDepth.maxDepth} (${report.aliasChainDepth.maxDepthToken})`,
);

if (componentCoverage) {
console.log("\n--- Component Coverage ---");
console.log(
` Registered components: ${componentCoverage.registeredComponentCount}`,
);
console.log(
` With tokens: ${componentCoverage.componentsWithTokens} (${componentCoverage.tokenCoveragePercent}%)`,
);
console.log(
` With schema: ${componentCoverage.componentsWithSchema} (${componentCoverage.schemaCoveragePercent}%)`,
);
console.log(
` With both: ${componentCoverage.componentsWithBoth}`,
);
}

console.log("\n--- Semantic Categories ---");
for (const [category, count] of Object.entries(report.semanticCategories)) {
console.log(` ${category.padEnd(24)} ${count}`);
}
}

program.parse();
Loading