Skip to content

Add extropy config test command to verify model connectivity #108

@DeveshParagiri

Description

@DeveshParagiri

Problem

There's no way to verify that configured models are working before running a pipeline. You only find out when a command fails mid-run, potentially after spending time/money on partial execution.

Proposed Solution

Add extropy config test command that:

  1. Pings each configured model (pipeline fast, pipeline strong, simulation fast, simulation strong) with a minimal request
  2. Reports success/failure for each
  3. Shows latency and any error messages

Example Output

$ extropy config test

Testing model connectivity...

Pipeline models:
  ✓ anthropic/claude-sonnet-4-6 (fast) — 342ms
  ✓ anthropic/claude-sonnet-4-6 (strong) — 358ms

Simulation models:
  ✓ azure/gpt-5-mini (fast) — 128ms
  ✓ azure/gpt-5-mini (strong) — 131ms

All models responding.
$ extropy config test

Testing model connectivity...

Pipeline models:
  ✓ anthropic/claude-sonnet-4-6 (fast) — 342ms
  ✗ anthropic/claude-sonnet-4-6 (strong) — Error: Invalid API key

Simulation models:
  ✓ azure/gpt-5-mini (fast) — 128ms
  ✓ azure/gpt-5-mini (strong) — 131ms

1 model failed. Check API keys and model availability.

Implementation Notes

  • Use the existing LLM client infrastructure
  • Send a trivial completion request (e.g., "Say 'ok'" with max_tokens=5)
  • Consider adding --model <model> flag to test a specific model
  • Exit code 0 if all pass, non-zero if any fail

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions