Session Usage Collection

[toubatbrian]

Summary by CodeRabbit

• New Features

  • Added adaptive interruption detection to identify when users interrupt agent responses with configurable thresholds.
  • Introduced enhanced turn handling configuration for improved control over interruption modes and endpointing behavior.
  • Added model provider and usage tracking for better visibility into which providers power your agent.
  • New user interruption detection events for improved conversation flow monitoring.

• Chores

  • Updated dependencies and removed ai-coustics plugin support.
  • Disabled test:examples workflow step.

[changeset-bot]

🦋 Changeset detected

Latest commit: 74e329b

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

• 🔍 Trigger a full review

📝 Walkthrough

Walkthrough

This PR introduces adaptive interruption detection infrastructure with HTTP and WebSocket transports, metrics collection and model usage aggregation, turn handling configuration system, and enhanced telemetry with provider/model metadata propagation across LLM, STT, and TTS services.

Changes

Cohort / File(s)	Summary
Interruption Detection Core agents/src/inference/interruption/defaults.ts, errors.ts, types.ts, utils.ts, utils.test.ts	Adds configuration constants, error class, type definitions, and utility functions (BoundedCache, probability estimation) for interruption inference system.
Interruption Detection Transports agents/src/inference/interruption/http_transport.ts, ws_transport.ts, interruption_cache_entry.ts	Implements HTTP and WebSocket-based interruption detection transports as TransformStreams with state management, retry logic, and structured response handling.
Interruption Detector & Stream agents/src/inference/interruption/interruption_detector.ts, interruption_stream.ts	Introduces AdaptiveInterruptionDetector for managing multiple streams and InterruptionStreamBase for audio transformation pipeline with transport abstraction.
Metrics & Usage Collection agents/src/metrics/base.ts, model_usage.ts, model_usage.test.ts, usage_collector.ts, index.ts	Adds MetricsMetadata type, model usage types (LLMModelUsage, TTSModelUsage, STTModelUsage), ModelUsageCollector for aggregation, and deprecation logging to UsageCollector.
Turn Handling Configuration agents/src/voice/turn_config/endpointing.ts, interruption.ts, turn_handling.ts, utils.ts, utils.test.ts	Introduces structured configuration for endpointing, interruption, and turn handling with migration utilities for legacy options.
Voice Agent Core Updates agents/src/voice/agent.ts, agent_activity.ts, agent_session.ts, events.ts	Integrates turn handling config, interruption detection, usage collection; adds UserInterruptionDetected/UserNonInterruptionDetected events; refactors option initialization.
Audio Recognition Integration agents/src/voice/audio_recognition.ts	Wires interruption detection into audio pipeline with transcript buffering, event emission, and trace attribute enrichment for STT model/provider.
Model Metadata & Telemetry agents/src/llm/llm.ts, realtime.ts, agents/src/stt/stt.ts, agents/src/tts/tts.ts, agents/src/voice/generation.ts, report.ts, agents/src/telemetry/trace_types.ts, traces.ts	Adds provider/model getters to LLM, STT, TTS; enriches metrics with token counts and metadata; adds trace attributes for latency (ttft, ttfb), interruption, and provider info; propagates usage through session reports.
Stream & API Enhancements agents/src/stream/stream_channel.ts	Extends StreamChannel with error type parameter, abort(), and addStreamInput() methods for graceful termination and external stream integration.
Configuration & Workflow .changeset/config.json, .github/workflows/test.yml, agents/package.json, examples/package.json, examples/src/basic_agent.ts	Formats changeset config, comments out example tests, adds ofetch dependency, removes @livekit/plugins-ai-coustics, updates basic agent to use BackgroundVoiceCancellation and adaptive interruption.

Sequence Diagram(s)

sequenceDiagram
    actor AudioSource
    participant AudioRecognition
    participant InterruptionStream
    participant HttpTransport
    participant InferenceAPI
    participant AudioAgent

    AudioSource->>AudioRecognition: audio frames
    AudioRecognition->>InterruptionStream: Int16Array chunks<br/>(with overlap tracking)
    
    InterruptionStream->>InterruptionStream: accumulate & resample<br/>to configured sample rate
    
    InterruptionStream->>HttpTransport: audio frames + threshold
    
    HttpTransport->>HttpTransport: batch frames, generate<br/>access token
    
    HttpTransport->>InferenceAPI: POST /bargein<br/>(audio, token, timestamp)
    
    InferenceAPI-->>HttpTransport: { is_bargein, probabilities,<br/>created_at, prediction_duration }
    
    HttpTransport->>HttpTransport: cache entry, compute<br/>probability via sliding window
    
    alt Interruption Detected (probability > threshold)
        HttpTransport->>InterruptionStream: InterruptionEvent
        InterruptionStream->>AudioAgent: emit interruption event<br/>(timestamp, duration, probability)
        AudioAgent->>AudioAgent: handle user interruption<br/>(pause generation, etc.)
    else No Interruption
        HttpTransport->>InterruptionStream: continue processing
    end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• Add AdaptiveInterruptionDetector #980: Directly related—modifies the same interruption-detection subsystem (AdaptiveInterruptionDetector, transport implementations, types, and defaults).
• Add agent activity interruption detector integration #991: Closely related—makes overlapping code-level changes to adaptive interruption integration, audio recognition, stream channel APIs, and voice event handling.
• fix basic_agent example #1003: Related—modifies the same example files (examples/package.json, examples/src/basic_agent.ts) to replace noise cancellation and remove @livekit/plugins-ai-coustics dependency.

Suggested reviewers

• chenghao-mou
• lukasIO

Poem

🐰 Hops with joy through interrupt flows!
HTTP streams and websockets glow,
Metrics dance, usage takes the stage,
Turn config tames the voice agent's cage,
Provider metadata marks the way—
A whisker-perfect PR today! ✨

🚥 Pre-merge checks | ❌ 3

❌ Failed checks (2 warnings, 1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	No pull request description was provided by the author, leaving reviewers without context about changes, motivation, or testing approach.	Provide a comprehensive PR description following the template, including what changes were made, why they were included, and how they were tested.
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.93% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	❓ Inconclusive	The title "[draft] Session Usage Collection" is vague and uses a [draft] prefix that suggests incomplete work, making it unclear what specific changes the PR contains.	Remove [draft] prefix and clarify the title to describe the main changes, such as 'Add session model usage collection and metrics aggregation' or 'Implement adaptive interruption detection integration'.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch brian/session-usage

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

agents-js/agents/src/inference/interruption/http_transport.ts

Lines 130 to 134 in d83d7b6

	baseUrl: options.baseUrl,
	timeout: options.timeout,
	maxRetries: options.maxRetries,
	token: await createAccessToken(options.apiKey, options.apiSecret),
	},

Skip JWT creation when credentials are optional

When the detector is configured with a non‑inference baseUrl, apiKey/apiSecret are allowed to be empty, but the HTTP transport still unconditionally calls createAccessToken and sends a LiveKit JWT. In that configuration this will either throw before the request is sent or attach an invalid Authorization header, causing all HTTP inference calls to fail for custom/unauthenticated backends. Consider bypassing token creation (or allowing caller‑provided auth) when credentials are intentionally unset.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[toubatbrian]

@lukasIO