Skip to content

Comments

feat: add cache compounding savings to rtk gain#261

Open
sahilmgandhi wants to merge 1 commit intortk-ai:masterfrom
sahilmgandhi:feat/cache-compounding-savings
Open

feat: add cache compounding savings to rtk gain#261
sahilmgandhi wants to merge 1 commit intortk-ai:masterfrom
sahilmgandhi:feat/cache-compounding-savings

Conversation

@sahilmgandhi
Copy link

@sahilmgandhi sahilmgandhi commented Feb 23, 2026

I think more importantly than just the immediate savings of rtk, it also offers compounding savings in the form of less cache thrashing. This is helpful both from a monetary perspective (cache writes/reads cost money) and also from a time perspective (fewer tokens from cache => less thinking => less time for output** (potentially))

Summary

  • Adds a "Cache Compounding Effect" section to rtk gain showing how direct token savings multiply through Claude Code's prompt caching
  • Scans JSONL session files to compute average session length, then applies the multiplier: 1.25 + 0.1 × avg_remaining_turns
  • Includes optional dollar amounts when ccusage is installed
  • Adds cache_compounding field to JSON/CSV export

How it works

When RTK removes tokens from command output, those tokens:

  1. Never get written to cache (avoids 1.25x input cost)
  2. Never get re-read from cache on every subsequent turn (avoids 0.1x per turn)

For a session with 336 avg turns (median position = 168 remaining), the multiplier is 1.25 + 0.1 × 168 = 18.06x.

Example output

Cache Compounding Effect
target/release/rtk gain
RTK Token Savings (Global Scope)
════════════════════════════════════════════════════════════

Total commands:    684
Input tokens:      2.0M
Output tokens:     255.4K
Tokens saved:      1.7M (87.1%)
Total exec time:   16m2s (avg 1.4s)
Efficiency meter: █████████████████████░░░ 87.1%

By Command
────────────────────────────────────────────────────────────────────────
  #  Command                   Count   Saved    Avg%    Time  Impact
────────────────────────────────────────────────────────────────────────
 1.  rtk go test ./server/...      2  866.3K  100.0%   41.9s  ██████████
 2.  rtk go test ./server/...      1  393.4K  100.0%   27.3s  █████░░░░░
 3.  rtk cargo clippy --al...     26  144.2K   92.9%   693ms  ██░░░░░░░░
 4.  rtk cargo test                8   62.2K   99.6%    2.3s  █░░░░░░░░░
 5.  rtk cargo test --all          5   38.2K   99.8%    1.6s  ░░░░░░░░░░
 6.  rtk find                    167   30.9K   79.1%     1ms  ░░░░░░░░░░
 7.  rtk git diff b0378a62...      4   26.7K   76.6%    53ms  ░░░░░░░░░░
 8.  rtk go test -run Test...      2   13.7K   99.9%   15.1s  ░░░░░░░░░░
 9.  rtk cargo test -- --n...      1    8.6K   99.9%    1.1s  ░░░░░░░░░░
10.  rtk ls                       34    8.3K   61.5%     7ms  ░░░░░░░░░░
────────────────────────────────────────────────────────────────────────

Cache Compounding Effect
──────────────────────────────────────────────────────────────
Direct savings:    1.7M
Avg session turns: 337 (from 48 sessions)
Avg remaining:     169
Cache multiplier:  18.12x  (1.25 + 0.1 x 169)
  ┌─────────────────────────────────────────────────────────┐
  │ Effective savings:   31.2M tokens  ($153.13)            │
  └─────────────────────────────────────────────────────────┘
How: Saved tokens avoid 1.25x cache write + 0.1x per
subsequent turn. Longer sessions = bigger multiplier.
How: Saved tokens avoid 1.25x cache write + 0.1x per
subsequent turn. Longer sessions = bigger multiplier.

Graceful degradation

  • No Claude Code sessions found → uses fallback estimate of 20 turns with "(model estimate)" label
  • No ccusage installed → omits dollar amount, shows install tip
  • Any failure → section silently omitted

Files changed

File Change
src/session_stats.rs NEW — session stats + compounding logic (8 unit tests)
src/cc_economics.rs Export WEIGHT_* constants as pub(crate), remove dead BILLION
src/main.rs Register mod session_stats
src/gain.rs Add display + JSON/CSV export, consolidate color helpers

Test plan

  • cargo fmt --all --check — clean
  • cargo clippy --all-targets — no new warnings
  • cargo test --all — 422 passed, 2 ignored
  • Manual: rtk gain shows new section with real session data
  • Manual: rtk gain --format json includes cache_compounding field

@sahilmgandhi sahilmgandhi force-pushed the feat/cache-compounding-savings branch from f0bf678 to 4aef192 Compare February 23, 2026 18:36
Show how direct token savings compound through Claude Code's prompt
caching. Saved tokens avoid a 1.25x cache write plus 0.1x cache read
on every subsequent turn, producing a multiplier based on average
session length (scanned from JSONL session files).

New section appears after the "By Command" table with multiplier,
effective savings, and optional dollar amounts (when ccusage installed).
Gracefully degrades: falls back to 20-turn estimate when no session
data, omits section entirely on failure.

- New module: src/session_stats.rs (8 unit tests)
- Export WEIGHT_* constants from cc_economics.rs
- Add cache_compounding field to JSON/CSV export
- Remove dead BILLION constant, consolidate color helpers
@sahilmgandhi sahilmgandhi force-pushed the feat/cache-compounding-savings branch from 4aef192 to fc2f85e Compare February 23, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant