fix(debug): use streaming to handle large JSONL files

[minislively]

Summary

The debug module was still using the readFile + split('\n') pattern which can cause RangeError: Invalid string length when processing large JSONL files (>512MB).

This is a follow-up to PR #706 which fixed the same issue in data-loader.ts but missed the debug.ts module.

Changes

• Export processJSONLFileByLine from data-loader.ts for reuse
• Refactor detectMismatches in debug.ts to use streaming instead of readFile
• Remove unused readFile import from debug.ts

Problem

When users have large usage datasets (1.5GB+ across many files), running debug commands like ccusage debug would fail with:

RangeError: Invalid string length
    at readFileHandle (node:internal/fs/promises:591:25)

Solution

Reuse the existing processJSONLFileByLine helper which uses Node.js streams (createReadStream + readline) to process files line-by-line without loading the entire file into memory.

Test Plan

• All existing tests pass (243 tests)
• Verified processJSONLFileByLine export doesn't break existing imports
• Debug module tests continue to work with streaming approach

---

Supersedes #815

🤖 Generated with Claude Code

Summary by CodeRabbit

• Refactor
  • Optimized file processing by implementing a streaming line-by-line approach for improved memory efficiency.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Exposed processJSONLFileByLine function from data-loader module as a public export. Updated debug.ts to replace file-reading and line-splitting logic with the streaming line processor, modifying control flow to use callbacks instead of direct iteration.

Changes

Cohort / File(s)	Summary
Streaming line processor export apps/ccusage/src/data-loader.ts	Exposed processJSONLFileByLine function as public export
Debug module refactoring apps/ccusage/src/debug.ts	Replaced readFile + splitLines with streaming processJSONLFileByLine callback-based processing; control flow tightened with early returns from callback instead of loop continuation

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• fix(ccusage): use streaming to handle large JSONL files #706: Implements streaming logic and uses processJSONLFileByLine across loaders; this PR exposes the function to enable that usage pattern.

Suggested reviewers

• ryoppippi

Poem

🐰 Whiskers twitch—the function's now in sight,
Export declared, made public and bright,
Streaming callbacks dance line by line,
File processing refactored, simply divine! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(debug): use streaming to handle large JSONL files' clearly and specifically describes the main change: refactoring the debug module to use streaming for processing large JSONL files instead of loading them entirely into memory.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

apps/ccusage/src/debug.ts (1)

137-145: Guard pricing lookup failures to prevent stream abort.

Result.unwrap() will throw on failures (unknown model, pricing fetch errors), which breaks the streaming and skips remaining lines in the file. Use Result.isFailure() with early return, consistent with the error handling pattern already established at lines 116-118 in this same function.

🔧 Suggested fix

-				const calculatedCost = await Result.unwrap(
-					fetcher.calculateCostFromTokens(data.message.usage, model),
-				);
+				const costResult = await fetcher.calculateCostFromTokens(data.message.usage, model);
+				if (Result.isFailure(costResult)) {
+					return;
+				}
+				const calculatedCost = costResult.value;