Skip to content

[BUG] Deprecated models are used for error handling during failed tool calls in Next environment #3626

@SamvelSimonyan

Description

@SamvelSimonyan

Steps to reproduce:

  1. Run an agent or pipeline in the Next environment.
  2. Make a tool call that fails.

Actual results:

  • When the tool call fails, low-tier LLM and sometimes high-tier LLM are called to debug the tool call as part of error handling.
  • Deprecated models like GPT-4o are used, causing issues as they do not work.
Image Image

Expected results:

  • Elitea should automatically choose the default low-tier model from the configuration for error handling.
  • If the default model is not selected or is unavailable, Elitea should select the next available model from the list, ensuring that tuned-off or deprecated models are not used.

Other info:

  • The issue is specific to error handling during failed tool calls in the Next environment.
  • Proper model selection logic should be implemented to avoid using deprecated models.

Metadata

Metadata

Labels

eng:aiAI/LLM engineering: orchestration, prompts, model behavior.feat:toolkitsToolkits feature: tool definitions, toolkit management, tool execution/runtime.rca:missing-functionalityRCA: missing functionality.rca:regressionRCA: regression from prior behavior.

Type

Projects

Status

Bugs

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions