feat: add cache compounding savings to rtk gain#261
Open
sahilmgandhi wants to merge 1 commit intortk-ai:masterfrom
Open
feat: add cache compounding savings to rtk gain#261sahilmgandhi wants to merge 1 commit intortk-ai:masterfrom
sahilmgandhi wants to merge 1 commit intortk-ai:masterfrom
Conversation
f0bf678 to
4aef192
Compare
Show how direct token savings compound through Claude Code's prompt caching. Saved tokens avoid a 1.25x cache write plus 0.1x cache read on every subsequent turn, producing a multiplier based on average session length (scanned from JSONL session files). New section appears after the "By Command" table with multiplier, effective savings, and optional dollar amounts (when ccusage installed). Gracefully degrades: falls back to 20-turn estimate when no session data, omits section entirely on failure. - New module: src/session_stats.rs (8 unit tests) - Export WEIGHT_* constants from cc_economics.rs - Add cache_compounding field to JSON/CSV export - Remove dead BILLION constant, consolidate color helpers
4aef192 to
fc2f85e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I think more importantly than just the immediate savings of
rtk, it also offers compounding savings in the form of less cache thrashing. This is helpful both from a monetary perspective (cache writes/reads cost money) and also from a time perspective (fewer tokens from cache => less thinking => less time for output** (potentially))Summary
rtk gainshowing how direct token savings multiply through Claude Code's prompt caching1.25 + 0.1 × avg_remaining_turnscache_compoundingfield to JSON/CSV exportHow it works
When RTK removes tokens from command output, those tokens:
For a session with 336 avg turns (median position = 168 remaining), the multiplier is
1.25 + 0.1 × 168 = 18.06x.Example output
Graceful degradation
Files changed
src/session_stats.rssrc/cc_economics.rsWEIGHT_*constants aspub(crate), remove deadBILLIONsrc/main.rsmod session_statssrc/gain.rsTest plan
cargo fmt --all --check— cleancargo clippy --all-targets— no new warningscargo test --all— 422 passed, 2 ignoredrtk gainshows new section with real session datartk gain --format jsonincludescache_compoundingfield