Token tracking race condition in threading-based async_execution

[Devasy]

Problem

The per-task token tracking feature has a race condition for tasks using threading-based async_execution (i.e., task.async_execution=True with synchronous crew.kickoff()).

Current Behavior

When multiple tasks with async_execution=True are executed by the same agent:

1. tokens_before is captured when the task is queued and submitted to the thread pool
2. Tasks run concurrently in threads
3. All concurrent tasks capture similar tokens_before values
4. tokens_after is captured sequentially in _process_async_tasks after calling future.result()
5. Later tasks get credited with tokens from earlier tasks that ran in parallel

Example

# Agent starts with 100 tokens used
- Task A queued: tokens_before = 100, starts running in thread
- Task B queued: tokens_before = 100, starts running in thread (nearly same time)
- Task A completes using 50 tokens (agent now at 150)
- Task B completes using 30 tokens (agent now at 180)

# Sequential processing in _process_async_tasks:
- Task A: tokens_after = 180, delta = 80 (WRONG - includes Task B's 30 tokens)
- Task B: tokens_after = 180, delta = 80 (WRONG - same as Task A)

Root Cause

The threading-based task.execute_async() uses concurrent.futures.ThreadPoolExecutor, which makes it difficult to wrap the execution to capture tokens immediately after completion within the thread.

Potential Solutions

1. Wrapper function in thread: Modify execute_async to accept a wrapper function that captures tokens immediately after task completion within the thread
2. Agent-level locking: Add a lock per agent to ensure only one async task per agent runs at a time (may impact performance)
3. Documentation: Document the limitation and recommend using akickoff() for accurate async task token tracking
4. Deprecation path: Consider deprecating threading-based async_execution in favor of native async

Related

• PR Feature: CrewAI Token Tracking Enhancement #4132 - Token tracking enhancement
• Fixed for native async (akickoff()) in commit 0f0538c
• Threading-based async still affected

Impact

• Medium: Affects token cost tracking accuracy when using async_execution=True with kickoff()
• Workaround: Use akickoff() instead, or avoid concurrent tasks from same agent

Additional Context

This issue only affects per-task token metrics. Crew-level and per-agent total metrics are still accurate as they're calculated after all tasks complete.

[Vidit-Ostwal]

I’m a bit skeptical about this. Each agent maintains its own LLM instance along with its own usage/metrics class, and these are independent of one another. Because of that, I don’t see a scenario where Task A and Task B would contend over a shared token count or end up with incorrect readings due to asynchronous execution.

Do you have a concrete example where you’ve observed this behavior, or is this concern primarily theoretical?

[Devasy]

Hi @Vidit-Ostwal, great question! You're absolutely right that each agent has its own LLM instance. The issue isn't about contention over a shared token count between agents. It's about accurate per-task attribution for a single agent executing multiple concurrent tasks.

Here's a concrete example:

The Race Condition Scenario

Setup: Agent A starts with 100 tokens used in its LLM instance.

Timeline with the unfixed code:

1. Task 1 queued: tokens_before = agent_a.llm.get_token_usage() = 100
2. Task 2 queued: tokens_before = agent_a.llm.get_token_usage() = 100 (both happen before any execution)
3. Task 1 executes in thread → Agent A's LLM records 50 new tokens (now at 150)
4. Task 2 executes in thread → Agent A's LLM records 30 new tokens (now at 180)
5. Task 1 completes: tokens_after = agent_a.llm.get_token_usage() = 180 (includes both tasks)
6. Task 2 completes: tokens_after = agent_a.llm.get_token_usage() = 180

Result:

• Task 1 delta: 180 - 100 = 80 tokens ❌ (should be 50, includes Task 2's tokens)
• Task 2 delta: 180 - 100 = 80 tokens ❌ (should be 30, includes Task 1's tokens)
• Actual: 50 + 30 = 80 tokens total, but we counted 160!

With Our Fix

We capture tokens_after immediately after each task completes within its thread, so:

• Task 1 completes → tokens_after = 150 → delta = 50 ✅
• Task 2 completes → tokens_after = 180 → delta = 30 ✅

So the issue is specifically about same-agent concurrent task attribution, not cross-agent issues. This matters for accurate cost tracking and per-task metrics when using kickoff() with async_execution=True.

[Vidit-Ostwal]

1. Is there any metric which calculates the tokens consumed per task?

[Vidit-Ostwal]

.get_token_usage(), is more for getting idea around how many token is used by that particular LLM instance.

[Vidit-Ostwal]

On a 2nd thought, it might be good idea to add a feature around how much tokens have been used per task.
If you are interested, you can try implementing this, I think you can open another feature request, share the approach and check with maintainers if it aligns with what they are thinking.
wdut?

[Devasy]

@Vidit-Ostwal Great question! The feature you're describing is actually already implemented in PR #4132

Our implementation provides both levels of granularity:

1. Per-Agent Total Tokens: Uses agent.llm.get_token_usage_summary() to get cumulative token usage across all tasks for each agent
2. Per-Task Token Breakdown: Uses delta calculation (tokens_after - tokens_before) to attribute specific token consumption to each task

The reason we need the delta approach for per-task metrics is that get_token_usage_summary() returns cumulative totals for the entire agent lifecycle. To break this down per-task, we capture the token state before and after each task execution, then calculate the difference.

This gives you:

• WorkflowTokenMetrics.per_agent_metrics - Total tokens consumed by each agent
• WorkflowTokenMetrics.per_task_metrics - Tokens consumed for each specific task execution

So both "per-agent totals" and "per-task breakdown" are available in the same PR! 🎯

[Vidit-Ostwal]

No, I think there’s a gap in this reasoning. If all agents share the same LLM instance, then calling agent.llm.get_token_usage_summary() from any agent will return the same result, since the underlying LLM state is shared.

One possible approach could be to pass agent_id and task_id into get_token_usage_summary(). The LLM could internally maintain a mapping of token usage by agent and task. Then, when task.get_token_usage_summary() is called, it would filter and return only the tokens consumed by that specific agent–task pair.

Just thinking out loud here—this might be a cleaner way to handle the problem.

[pdaxt]

@Vidit-Ostwal Good point about the shared LLM instance challenge. Here's a pattern that works well in production:

Thread-Local Token Accumulator

Instead of modifying the LLM class, wrap token tracking at the execution layer:

import threading
from contextlib import contextmanager
from dataclasses import dataclass, field

@dataclass
class TaskTokenContext:
    task_id: str
    agent_id: str
    tokens: dict = field(default_factory=lambda: {'input': 0, 'output': 0})

# Thread-local storage for current task context
_task_context = threading.local()

@contextmanager
def track_task_tokens(task_id: str, agent_id: str):
    """Context manager that captures tokens for this specific task"""
    _task_context.current = TaskTokenContext(task_id, agent_id)
    try:
        yield _task_context.current
    finally:
        # Store results before clearing
        result = _task_context.current
        _task_context.current = None
        return result

# Patch the LLM callback to route tokens to current task
def on_llm_token_usage(input_tokens: int, output_tokens: int):
    ctx = getattr(_task_context, 'current', None)
    if ctx:
        ctx.tokens['input'] += input_tokens
        ctx.tokens['output'] += output_tokens

Why this works:

1. Each thread has its own _task_context
2. No shared state between concurrent tasks
3. LLM instance stays unchanged - we intercept at the callback level
4. Works with async_execution=True since each task runs in its own thread

This avoids the race condition entirely because tokens are accumulated per-thread, not per-agent.

Happy to submit a PR if this approach aligns with the codebase architecture.

[Devasy]

@Vidit-Ostwal @pdaxt Great observations! Let me clarify the design assumptions in our PR.

Current Design Assumption

PR #4132 (#4132) assumes each agent has its own LLM instance, which is the typical usage pattern in CrewAI:

# Typical usage - separate LLM instances
agent1 = Agent(llm=LLM(model="gpt-4"), role="Researcher", ...)
agent2 = Agent(llm=LLM(model="gpt-4"), role="Writer", ...)

Each BaseAgent has its own llm field (base_agent.py#L149), and when initialized normally, agents get separate LLM instances even if using the same model.

The Edge Case You Identified

However, @Vidit-Ostwal is correct about the shared LLM instance problem:

# Edge case - shared LLM instance
shared_llm = LLM(model="gpt-4")
agent1 = Agent(llm=shared_llm, role="Researcher", ...)
agent2 = Agent(llm=shared_llm, role="Writer", ...)

In this scenario, calling agent.llm.get_token_usage_summary() would return cumulative tokens for both agents, breaking per-agent attribution.

Solution Approach

@pdaxt's thread-local pattern is interesting, but I think there's a simpler solution that aligns better with CrewAI's architecture:

Document this as a usage constraint: Users must not share LLM instances across agents if they want accurate per-agent metrics. This is already the recommended pattern in CrewAI.

Alternatively, we could add a validation check in the Crew initialization:

# Check for shared LLM instances
llm_instances = {}
for agent in self.agents:
    llm_id = id(agent.llm)
    if llm_id in llm_instances:
        raise ValueError(
            f"Agents '{agent.role}' and '{llm_instances[llm_id]}' share the same LLM instance. "
            "For accurate token tracking, each agent must have its own LLM instance."
        )
    llm_instances[llm_id] = agent.role

Thoughts? Should we add this validation or just document the constraint?

[pdaxt]

@Devasy Thanks for the detailed breakdown! You're right that the typical CrewAI usage pattern creates separate LLM instances per agent, which makes the per-agent tracking straightforward.

On the Validation Check Approach

I like the validation option - it's defensive and catches a subtle footgun early. One enhancement:

def _validate_llm_instances(self):
    """Validate LLM instance uniqueness for accurate token tracking."""
    llm_instances = {}
    warnings = []
    
    for agent in self.agents:
        llm_id = id(agent.llm)
        if llm_id in llm_instances:
            # Option: warn instead of error for backwards compat
            warnings.append(
                f"Agents '{agent.role}' and '{llm_instances[llm_id]}' share an LLM instance. "
                f"Per-agent token metrics will be aggregated for these agents."
            )
        llm_instances[llm_id] = agent.role
    
    if warnings:
        logger.warning("\n".join(warnings))
        self._has_shared_llm = True  # Flag for downstream handling

Why warn instead of error: Some users intentionally share LLM instances to reduce memory (e.g., running 10 agents with the same model). Breaking them would be a regression. But we should definitely log it so they know per-agent metrics won't be accurate.

Hybrid Approach for Per-Task Metrics

The thread-local pattern I mentioned is actually orthogonal to the per-agent question - it solves per-task attribution regardless of LLM sharing:

Scenario	Per-Agent (your PR)	Per-Task (thread-local)
Separate LLM instances	✅ Works	✅ Works
Shared LLM instance	❌ Aggregated	✅ Still works
Concurrent async_execution	✅ Works	✅ No race condition

If CrewAI wants per-task granularity (which users are asking for in this thread), the thread-local approach complements your per-agent tracking nicely.

Suggestion

1. Merge your PR Feature: CrewAI Token Tracking Enhancement #4132 with the validation warning added
2. Future PR could add optional per-task tracking using thread-local context for users who need that granularity

Happy to help with either - just let me know which direction the maintainers prefer. This is a well-designed feature and I'd love to see it land! 🚀

[Devasy]

@pdaxt Really appreciate your detailed analysis! You're absolutely right about the hybrid approach and the warning vs error trade-off.

Clarification

I apologize for suggesting validation changes - that's for the maintainers to decide. Let me just clarify what PR #4132 currently implements:

Our PR already provides per-task token tracking:

• WorkflowTokenMetrics.per_task_metrics - Dictionary keyed by {task_id}_{task_name}_{agent_name}
• Each entry shows exact tokens consumed for that specific task execution
• Works correctly with async_execution=True via our threading/async wrapper approach

Your thread-local pattern is indeed a complementary approach that could serve as an alternative implementation, especially for the shared LLM scenario you highlighted.

Summary Table (Updated)

Feature	PR #4132 Status
Per-agent token totals	✅ Implemented
Per-task token breakdown	✅ Implemented
Concurrent async execution	✅ Fixed with thread wrappers
Shared LLM instances	⚠️ Works only if agents have separate LLM instances

I'll defer to the maintainers on whether to add validation/warnings or explore the thread-local approach for shared LLM scenarios. Thanks for the constructive feedback! 🙏