Skip to content

Conversation

@shotarok
Copy link

@shotarok shotarok commented Nov 5, 2025

Add support for instrumentation of portkey_ai.AsyncPortkey


Note

Adds async instrumentation for portkey_ai.AsyncPortkey by wrapping AsyncCompletions.create, plus tests and cassettes.

  • Instrumentation (Portkey):
    • Wrap AsyncCompletions.create in portkey_ai.api_resources.apis.{chat_complete,generation} with _AsyncCompletionsWrapper and track original for uninstrumentation.
    • Add _AsyncCompletionsWrapper to handle async spans, request/response attribute extraction, and span name AsyncCompletions.
    • Extend instrumentor slots to include _original_async_completions_create.
  • Tests:
    • Add async tests test_async_chat_completion and test_async_prompt_template asserting span name/attributes.
    • Add corresponding VCR cassettes tests/cassettes/test_async_chat_completion.yaml and test_async_prompt_template.yaml.

Written by Cursor Bugbot for commit fac30e1. This will update automatically on new commits. Configure here.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2025

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@shotarok
Copy link
Author

shotarok commented Nov 5, 2025

I have read the CLA Document and I hereby sign the CLA

github-actions bot added a commit that referenced this pull request Nov 5, 2025
@shotarok shotarok marked this pull request as ready for review November 5, 2025 18:43
@shotarok shotarok requested a review from a team as a code owner November 5, 2025 18:43
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Nov 5, 2025
portkey_module.Completions.create = self._original_completions_create

if self._original_async_completions_create is not None:
portkey_module.AsyncCompletions.create = self._original_async_completions_create
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Incomplete uninstrumentation leaves generation wrap active

The _uninstrument method does not properly restore AsyncCompletions.create from the generation module. During instrumentation (lines 64-68), AsyncCompletions.create is wrapped in both portkey_ai.api_resources.apis.chat_complete and portkey_ai.api_resources.apis.generation modules. However, the uninstrument method only imports and restores from the chat_complete module (lines 71-76), leaving the generation module's AsyncCompletions.create method wrapped even after uninstrumentation. This will cause the instrumentation to remain active for async completions called through the generation module even after calling uninstrument().

Fix in Cursor Fix in Web

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is true, we would need to restore the sync function, in addition to the async function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant