Skip to content

feat(otel, core): record turn TTFT and TTFM metrics in codex-core#13630

Merged
owenlin0 merged 2 commits intomainfrom
owen/add_ttft_ttfm_core_metrics
Mar 6, 2026
Merged

feat(otel, core): record turn TTFT and TTFM metrics in codex-core#13630
owenlin0 merged 2 commits intomainfrom
owen/add_ttft_ttfm_core_metrics

Conversation

@owenlin0
Copy link
Collaborator

@owenlin0 owenlin0 commented Mar 5, 2026

Summary

This adds turn-level latency metrics for the first model output and the first completed agent message.

  • codex.turn.ttft.duration_ms starts at turn start and records on the first output signal we see from the model. That includes normal assistant text, reasoning deltas, and non-text outputs like tool-call items.
  • codex.turn.ttfm.duration_ms also starts at turn start, but it records when the first agent message finishes streaming rather than when its first delta arrives.

Implementation notes

The timing is tracked in codex-core, not app-server, so the definition stays consistent across CLI, TUI, and app-server clients.

I reused the existing turn lifecycle boundary that already drives codex.turn.e2e_duration_ms, stored the turn start timestamp in turn state, and record each metric once per turn.

I also wired the new metric names into the OTEL runtime metrics summary so they show up in the same in-memory/debug snapshot path as the existing timing metrics.

@owenlin0 owenlin0 marked this pull request as ready for review March 5, 2026 22:16
@owenlin0
Copy link
Collaborator Author

owenlin0 commented Mar 5, 2026

@codex review

Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

tokio::spawn(
async move {

P2 Badge Initialize turn timing state before spawning the task

The task is spawned first, and turn state is marked started only later in register_new_active_task after an await on total_token_usage. A fast stream can emit first output before mark_turn_started, so record_turn_ttft/ttfm returns None and misses (or delays) the first-latency metric for that turn.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@owenlin0 owenlin0 force-pushed the owen/add_ttft_ttfm_core_metrics branch from d7afd1f to 98d86b9 Compare March 5, 2026 22:47
@owenlin0
Copy link
Collaborator Author

owenlin0 commented Mar 5, 2026

@codex review

@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Collaborator

@pakrym-oai pakrym-oai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we offload more logic from codex.rs?

@owenlin0 owenlin0 enabled auto-merge (squash) March 6, 2026 18:01
@owenlin0 owenlin0 disabled auto-merge March 6, 2026 18:23
@owenlin0 owenlin0 merged commit 3449e00 into main Mar 6, 2026
29 of 31 checks passed
@owenlin0 owenlin0 deleted the owen/add_ttft_ttfm_core_metrics branch March 6, 2026 18:23
@github-actions github-actions bot locked and limited conversation to collaborators Mar 6, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants