Add file-based usage metrics for local runs (#110)

[neoneye]

Summary

• Add file-based usage metrics (usage_metrics.jsonl) for local runs that don't have database access
• Records per-LLM-call metrics via llama_index instrumentation: model (with provider prefix), tokens (input/output/thinking), duration, cost, and success/failure
• Failures are recorded separately by LLMExecutor since instrumentation end events aren't emitted on failure
• Pipeline sets the metrics path at start and clears it after completion
• usage_metrics.jsonl is excluded from pipeline progress calculation

Example output

{"timestamp": "2026-03-10T13:36:48.250446", "success": true, "model": "Google AI Studio:google/gemini-2.0-flash-001", "duration_seconds": 4.879, "input_tokens": 5316, "output_tokens": 643, "cost_usd": 0.0007888}
{"timestamp": "2026-03-10T13:36:53.554864", "success": true, "model": "Google:google/gemini-2.0-flash-001", "duration_seconds": 5.237, "input_tokens": 8877, "output_tokens": 562, "cost_usd": 0.0011125}

Code quality improvements

• Move usage_metrics import to top-level so bad imports fail hard on startup
• Remove redundant try/except — record_usage_metric handles errors internally
• Warn (not debug-log) when metrics path is unset or write fails
• Document set_usage_metrics_path(None) teardown in module docstring
• Place success field before model in JSONL output for easier error skimming

Bug fixes

• Resume for legacy plans: Frontend and MCP resume checks incorrectly rejected plans created before pipeline_version was stamped into parameters (comparing None != PIPELINE_VERSION). The worker-side check against the actual snapshot metadata file is the real safety gate
• Heartbeat crash: A corrupted psycopg2 connection during WorkerItem.upsert_heartbeat() was propagating up and killing Luigi tasks. Wrapped in try/except with session rollback since the heartbeat is just a liveness signal

Files changed

• worker_plan/worker_plan_internal/llm_util/usage_metrics.py — core module for file-based metric recording
• worker_plan/worker_plan_internal/llm_util/track_activity.py — records successful calls via _record_file_usage_metric()
• worker_plan/worker_plan_internal/llm_util/llm_executor.py — records failed calls only
• worker_plan/worker_plan_internal/plan/run_plan_pipeline.py — sets/clears metrics path around pipeline execution
• worker_plan/worker_plan_api/filenames.py — add USAGE_METRICS_JSONL constant
• frontend_multi_user/src/app.py — fix resume version check for legacy plans
• mcp_cloud/db_queries.py — fix resume version check for legacy plans
• worker_plan_database/app.py — protect pipeline from heartbeat failures
• docs/proposals/110-usage-metrics-local-runs.md — mark as implemented
• docs/proposals/111-promising-directions.md — mark Mcp example plans #110 as complete

Test plan

• Run a local pipeline and verify usage_metrics.jsonl is created with per-call token counts and cost
• Verify the model field includes the provider prefix (e.g. Google AI Studio:google/gemini-2.0-flash-001)
• Verify metrics recording does not block pipeline on write failure
• Stop a plan, then resume it — verify usage_metrics.jsonl is created on resume
• Resume a legacy plan (no pipeline_version in parameters) — verify it is not rejected
• Verify a heartbeat database error does not crash the pipeline

🤖 Generated with Claude Code