docs: Add OpenTelemetry GenAI semantic conventions setup guide by tonybaloney · Pull Request #529 · github/copilot-sdk

tonybaloney · 2026-02-22T01:01:56Z

Summary

This PR adds comprehensive documentation for setting up OpenTelemetry tracing with GenAI semantic conventions for the Copilot SDK.

What's Included

Installation instructions for various observability backends (console, Azure Monitor, OTLP)
Basic setup guide with step-by-step code examples
Complete semantic conventions mapping following OpenTelemetry GenAI Semantic Conventions v1.34.0
Detailed reference tables for:
- Span attributes (agent, request, response, usage, thread, message, tool, error)
- Span events (messages, system instructions, conversation items)
- Operation names and span naming conventions
- Metrics (operation duration, token usage)
Content recording configuration for controlling sensitive data capture
Custom function tracing with the @trace_function decorator
Custom attributes via span processors
Complete working example
Troubleshooting guide

Source

This documentation is derived from the production implementation in Azure SDK for Python, specifically from:

sdk/ai/azure-ai-agents/
sdk/ai/azure-ai-projects/
sdk/ai/azure-ai-inference/

References

This guide should help developers implement standardized observability for their AI agent applications using the Copilot SDK.

Copilot

Pull request overview

This PR attempts to add OpenTelemetry tracing documentation for the Copilot SDK, but the content is actually documentation for the Azure SDK for Python's AI agents implementation, not for the GitHub Copilot SDK.

Changes:

Adds a new documentation file docs/opentelemetry-genai-setup.md with 480 lines of OpenTelemetry setup instructions

Comments suppressed due to low confidence (2)

docs/opentelemetry-genai-setup.md:401

This complete example imports Azure SDK packages (azure.ai.agents.telemetry, azure.ai.projects, azure.identity) and uses Azure-specific APIs (AIProjectClient, agents_client) that do not exist in the GitHub Copilot SDK. The Copilot SDK uses CopilotClient and communicates with sessions, not agents.

## Complete Example

```python
import os
from azure.core.settings import settings
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from azure.ai.agents.telemetry import AIAgentsInstrumentor
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

# Enable OpenTelemetry
settings.tracing_implementation = "opentelemetry"

# Setup tracing to console
span_exporter = ConsoleSpanExporter()
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)

# Instrument agents
AIAgentsInstrumentor().instrument()

# Create client
project_client = AIProjectClient(
    endpoint=os.environ["PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)

# Execute with tracing
with tracer.start_as_current_span("agent-interaction"):
    with project_client:
        agents_client = project_client.agents
        
        # Create agent
        agent = agents_client.create_agent(
            model=os.environ["MODEL_DEPLOYMENT_NAME"],
            name="my-agent",
            instructions="You are a helpful agent"
        )
        
        # Create thread and message
        thread = agents_client.threads.create()
        message = agents_client.messages.create(
            thread_id=thread.id,
            role="user",
            content="Hello, tell me a joke"
        )
        
        # Run agent
        run = agents_client.runs.create_and_process(
            thread_id=thread.id,
            agent_id=agent.id
        )
        
        print(f"Run completed with status: {run.status}")
        
        # Clean up
        agents_client.delete_agent(agent.id)

**docs/opentelemetry-genai-setup.md:67**
* This code example imports Azure-specific modules (`azure.core.settings`, `azure.ai.agents.telemetry`) that do not exist in the GitHub Copilot SDK. The Copilot SDK does not provide `AIAgentsInstrumentor` or any OpenTelemetry instrumentation classes.

from azure.core.settings import settings
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter

# Set the tracing implementation to OpenTelemetry
settings.tracing_implementation = "opentelemetry"

# Setup tracer provider
span_exporter = ConsoleSpanExporter()
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)

2. Instrument Your Application

from azure.ai.agents.telemetry import AIAgentsInstrumentor

# Enable automatic instrumentation
AIAgentsInstrumentor().instrument()

# Your agent code here...

# Optional: Disable instrumentation when done
AIAgentsInstrumentor().uninstrument()

</details>

Copilot · 2026-02-22T01:04:55Z

docs/opentelemetry-genai-setup.md

+    # Your agent code here
+    agent = agents_client.create_agent(...)
+    thread = agents_client.threads.create()
+    # ... more agent operations


This Azure Monitor setup code uses Azure SDK classes that are not part of the GitHub Copilot SDK. The example references agents_client which does not exist in the Copilot SDK API.

This issue also appears on line 340 of the same file.

Suggested change

# Your agent code here

agent = agents_client.create_agent(...)

thread = agents_client.threads.create()

# ... more agent operations

# Your Copilot SDK agent operations go here, for example:

# from github_copilot_sdk import CopilotClient

# client = CopilotClient(...)

# response = client.send_message("Hello, world!")

pass

Copilot · 2026-02-22T01:04:55Z

docs/opentelemetry-genai-setup.md

+| `gen_ai.system` | The GenAI system identifier | `az.ai.agents` |
+| `gen_ai.operation.name` | Operation being performed | `create_agent`, `start_thread_run`, `execute_tool` |
+| `gen_ai.provider.name` | Provider name | `microsoft.foundry` |
+| `az.namespace` | Azure namespace | `Microsoft.CognitiveServices` |
+| `server.address` | Server endpoint | `https://...` |
+| `server.port` | Server port (if not 443) | `443` |
+


Copilot · 2026-02-22T01:04:56Z

docs/opentelemetry-genai-setup.md

+## Tracing Custom Functions
+
+Use the `@trace_function` decorator to trace your own functions:
+
+```python
+from azure.ai.agents.telemetry import trace_function
+
+@trace_function(name="my_custom_function")
+def my_function(param1, param2):
+    # Your function code
+    return result
+```
+
+The decorator automatically captures:
+- Function parameters as `code.function.parameter.<parameter_name>`
+- Return values as `code.function.return.value`
+- Supports: str, int, float, bool, list, dict, tuple, set
+
+**Note:** All parameters and return values are traced when using `@trace_function`, regardless of content recording settings.


The @trace_function decorator from azure.ai.agents.telemetry does not exist in the GitHub Copilot SDK. This is an Azure SDK feature that is not available in this repository.

Copilot · 2026-02-22T01:04:56Z

docs/opentelemetry-genai-setup.md

+## Troubleshooting
+
+### Tracing Not Working
+
+1. Ensure OpenTelemetry is properly initialized before creating agents
+2. Verify `settings.tracing_implementation = "opentelemetry"` is set
+3. Check that AIAgentsInstrumentor is instrumented: `AIAgentsInstrumentor().is_instrumented()`
+
+### No Span Output
+
+1. Verify tracer provider is set: `trace.set_tracer_provider(tracer_provider)`
+2. Ensure span processor is added to tracer provider
+3. Call `exporter.force_flush()` before application exit
+
+### Content Not Recorded
+
+1. Check environment variable: `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true`
+2. Verify content recording is enabled: `AIAgentsInstrumentor().is_content_recording_enabled()`
+3. Note: `@trace_function` always traces content regardless of this setting
+
+### Missing Attributes
+
+Some attributes are conditional:
+- `server.port` - only added if port is not 443
+- Many request/response attributes - only added if the value is provided
+- Content-related attributes - only added if content recording is enabled


The troubleshooting section references Azure SDK-specific APIs like AIAgentsInstrumentor().is_instrumented() and AIAgentsInstrumentor().is_content_recording_enabled() which do not exist in the GitHub Copilot SDK.

This issue also appears on line 38 of the same file.

Copilot · 2026-02-22T01:04:56Z

docs/opentelemetry-genai-setup.md

+# OpenTelemetry GenAI Semantic Conventions Setup Guide
+
+This guide explains how to set up OpenTelemetry tracing with GenAI semantic conventions for the Copilot SDK, based on the implementation in Azure SDK for Python.
+
+## Overview
+
+The Azure SDK for Python implements comprehensive OpenTelemetry tracing for AI agent operations using the [OpenTelemetry GenAI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). This provides standardized observability for agent interactions, token usage, tool calls, and conversation flow.


This documentation describes OpenTelemetry setup for the Azure SDK for Python (azure.ai.agents), not for the GitHub Copilot SDK. The GitHub Copilot SDK does not have any OpenTelemetry integration or Azure-specific components. This appears to be documentation from a different project that has been incorrectly added to this repository.

The Copilot SDK is a wrapper around the GitHub Copilot CLI and operates via JSON-RPC communication. It does not import or depend on any Azure packages like azure.ai.agents, azure.ai.projects, or azure.core.settings.

Copilot · 2026-02-22T01:04:56Z

docs/opentelemetry-genai-setup.md

+## Content Recording
+
+Content recording controls whether message contents, tool parameters, and return values are captured in traces. This data may include sensitive information.
+
+### Enable Content Recording
+
+Set the environment variable (recommended by OpenTelemetry):
+```bash
+export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true
+```
+
+Or use the legacy Azure-specific variable:
+```bash
+export AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED=true
+```
+
+**Important:** If either variable is set to `false`, content recording will be disabled regardless of the other's value.
+
+### Checking Content Recording Status
+
+```python
+from azure.ai.agents.telemetry import AIAgentsInstrumentor
+
+if AIAgentsInstrumentor().is_content_recording_enabled():
+    print("Content recording is enabled")
+```


The environment variables mentioned (OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED) are specific to the Azure SDK for Python and are not used or recognized by the GitHub Copilot SDK.

Copilot · 2026-02-22T01:04:56Z

docs/opentelemetry-genai-setup.md

+Install the required OpenTelemetry packages:
+
+```bash
+pip install opentelemetry-sdk azure-core-tracing-opentelemetry
+```
+
+### Optional: Export to observability backends
+
+For console output:
+```bash
+# No additional packages needed - uses ConsoleSpanExporter
+```
+
+For Azure Monitor:
+```bash
+pip install azure-monitor-opentelemetry
+```
+
+For OTLP-compatible backends (like Aspire Dashboard):
+```bash
+pip install opentelemetry-exporter-otlp-proto-grpc
+```


The installation instructions reference Azure-specific packages (azure-core-tracing-opentelemetry, azure-monitor-opentelemetry) that are not dependencies of the GitHub Copilot SDK. The Copilot SDK does not integrate with Azure's tracing infrastructure.

Add comprehensive documentation for instrumenting Copilot SDK applications with OpenTelemetry GenAI semantic conventions. The guide includes: - Installation and basic setup for OpenTelemetry - Complete SessionEventType to GenAI attribute mapping - Detailed examples for ASSISTANT_USAGE, TOOL_EXECUTION events - Full working code example with proper span management - Content recording guidelines (opt-in for sensitive data) - MCP tool conventions - Metrics export - Azure Monitor integration - Best practices and troubleshooting This enables Copilot SDK users to add standardized observability following OpenTelemetry GenAI Semantic Conventions v1.34.0.

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (11)

docs/opentelemetry-instrumentation.md:249

The session.send() method returns a message ID string, not an async iterator of events. You cannot use "async for event in session.send(...)". Instead, events must be received by registering a handler with session.on(). The correct pattern is: session.on(lambda event: handle_event(event)) followed by await session.send({"prompt": "..."}). All examples using async for with session.send() are incorrect and need to be rewritten.

async for event in session.send("Tell me a joke"):
    if event.type == SessionEventType.ASSISTANT_MESSAGE and event.data:
        if event.data.content:
            # Add as a span event (opt-in for content recording)
            span.add_event(
                "gen_ai.output.messages",
                attributes={
                    "gen_ai.event.content": json.dumps({
                        "role": "assistant",
                        "content": event.data.content
                    })
                }
            )

**docs/opentelemetry-instrumentation.md:349**
* The complete example does not properly clean up resources. According to the Python SDK documentation and samples, both the session and client should be explicitly cleaned up. Add a finally block that calls await session.destroy() if the session exists, and await client.stop() to properly shut down the client and CLI process.

async def invoke_agent(prompt: str):
"""Invoke agent with full OpenTelemetry instrumentation."""

# Create main span
span_attrs = {
    "gen_ai.operation.name": "invoke_agent",
    "gen_ai.provider.name": "github.copilot",
    "gen_ai.agent.name": "example-agent",
    "gen_ai.request.model": "gpt-5",
}

span = tracer.start_span(
    name="invoke_agent example-agent",
    kind=SpanKind.CLIENT,
    attributes=span_attrs
)
token = context.attach(trace.set_span_in_context(span))
tool_spans = {}

try:
    client = CopilotClient(SessionConfig(model="gpt-5"))
    async with client.create_session() as session:
        async for event in session.send(prompt):
            data = event.data
            
            # Handle usage events
            if event.type == SessionEventType.ASSISTANT_USAGE and data:
                if data.model:
                    span.set_attribute("gen_ai.response.model", data.model)
                if data.input_tokens is not None:
                    span.set_attribute("gen_ai.usage.input_tokens", int(data.input_tokens))
                if data.output_tokens is not None:
                    span.set_attribute("gen_ai.usage.output_tokens", int(data.output_tokens))
            
            # Handle tool execution
            elif event.type == SessionEventType.TOOL_EXECUTION_START and data:
                call_id = data.tool_call_id or str(uuid.uuid4())
                tool_name = data.tool_name or "unknown"
                
                tool_attrs = {
                    "gen_ai.tool.name": tool_name,
                    "gen_ai.operation.name": "execute_tool",
                    "gen_ai.tool.call.id": call_id,
                }
                
                tool_span = tracer.start_span(
                    name=f"execute_tool {tool_name}",
                    kind=SpanKind.CLIENT,
                    attributes=tool_attrs
                )
                tool_token = context.attach(trace.set_span_in_context(tool_span))
                tool_spans[call_id] = (tool_span, tool_token)
            
            elif event.type == SessionEventType.TOOL_EXECUTION_COMPLETE and data:
                call_id = data.tool_call_id
                entry = tool_spans.pop(call_id, None) if call_id else None
                if entry:
                    tool_span, tool_token = entry
                    context.detach(tool_token)
                    tool_span.end()
            
            # Capture final message
            elif event.type == SessionEventType.ASSISTANT_MESSAGE and data:
                if data.content:
                    print(f"Assistant: {data.content}")
    
    span.set_attribute("gen_ai.response.finish_reasons", ["stop"])

except Exception as e:
    span.set_attribute("error.type", type(e).__name__)
    raise
finally:
    # Clean up any unclosed tool spans
    for call_id, (tool_span, tool_token) in tool_spans.items():
        tool_span.set_attribute("error.type", "stream_aborted")
        context.detach(tool_token)
        tool_span.end()
    
    context.detach(token)
    span.end()

**docs/opentelemetry-instrumentation.md:292**
* The session.send() method expects a MessageOptions dictionary with a "prompt" key, not a plain string. According to the Python SDK implementation, all these send() calls should use the format: session.send({"prompt": prompt}) instead of session.send(prompt).

        async for event in session.send(prompt):

**docs/opentelemetry-instrumentation.md:171**
* The session.send() method expects a MessageOptions dictionary with a "prompt" key, not a plain string. This should be: session.send({"prompt": "What's the weather?"}) instead of session.send("What's the weather?").

async for event in session.send("What's the weather?"):

**docs/opentelemetry-instrumentation.md:223**
* The session.send() method returns a message ID string, not an async iterator of events. You cannot use "async for event in session.send(...)". Instead, events must be received by registering a handler with session.on(). The correct pattern is: session.on(lambda event: handle_event(event)) followed by await session.send({"prompt": "..."}). All examples using async for with session.send() are incorrect and need to be rewritten.

async for event in session.send("What's the weather?"):
data = event.data

if event.type == SessionEventType.TOOL_EXECUTION_START and data:
    call_id = data.tool_call_id or str(uuid.uuid4())
    tool_name = data.tool_name or "unknown"
    
    tool_attrs = {
        "gen_ai.tool.name": tool_name,
        "gen_ai.operation.name": "execute_tool",
    }
    
    if call_id:
        tool_attrs["gen_ai.tool.call.id"] = call_id
    
    # Optional: include tool arguments (may contain sensitive data)
    if data.arguments is not None:
        try:
            tool_attrs["gen_ai.tool.call.arguments"] = json.dumps(data.arguments)
        except Exception:
            tool_attrs["gen_ai.tool.call.arguments"] = str(data.arguments)
    
    tool_span = tracer.start_span(
        name=f"execute_tool {tool_name}",
        kind=SpanKind.CLIENT,
        attributes=tool_attrs
    )
    tool_token = context.attach(trace.set_span_in_context(tool_span))
    tool_spans[call_id] = (tool_span, tool_token)

elif event.type == SessionEventType.TOOL_EXECUTION_COMPLETE and data:
    call_id = data.tool_call_id
    entry = tool_spans.pop(call_id, None) if call_id else None
    
    if entry:
        tool_span, tool_token = entry
        
        # Optional: include tool result (may contain sensitive data)
        if data.result is not None:
            try:
                result_str = json.dumps(data.result)
            except Exception:
                result_str = str(data.result)
            # Truncate to 512 chars to avoid huge spans
            tool_span.set_attribute("gen_ai.tool.call.result", result_str[:512])
        
        # Mark as error if tool failed
        if hasattr(data, "success") and data.success is False:
            tool_span.set_attribute("error.type", "tool_error")
        
        context.detach(tool_token)
        tool_span.end()

docs/opentelemetry-instrumentation.md:349

The session.send() method returns a message ID string, not an async iterator of events. You cannot use "async for event in session.send(...)". Instead, events must be received by registering a handler with session.on(). The entire example needs to be rewritten to use the correct event handling pattern with session.on() to register handlers before calling send().

            async for event in session.send(prompt):
                data = event.data
                
                # Handle usage events
                if event.type == SessionEventType.ASSISTANT_USAGE and data:
                    if data.model:
                        span.set_attribute("gen_ai.response.model", data.model)
                    if data.input_tokens is not None:
                        span.set_attribute("gen_ai.usage.input_tokens", int(data.input_tokens))
                    if data.output_tokens is not None:
                        span.set_attribute("gen_ai.usage.output_tokens", int(data.output_tokens))
                
                # Handle tool execution
                elif event.type == SessionEventType.TOOL_EXECUTION_START and data:
                    call_id = data.tool_call_id or str(uuid.uuid4())
                    tool_name = data.tool_name or "unknown"
                    
                    tool_attrs = {
                        "gen_ai.tool.name": tool_name,
                        "gen_ai.operation.name": "execute_tool",
                        "gen_ai.tool.call.id": call_id,
                    }
                    
                    tool_span = tracer.start_span(
                        name=f"execute_tool {tool_name}",
                        kind=SpanKind.CLIENT,
                        attributes=tool_attrs
                    )
                    tool_token = context.attach(trace.set_span_in_context(tool_span))
                    tool_spans[call_id] = (tool_span, tool_token)
                
                elif event.type == SessionEventType.TOOL_EXECUTION_COMPLETE and data:
                    call_id = data.tool_call_id
                    entry = tool_spans.pop(call_id, None) if call_id else None
                    if entry:
                        tool_span, tool_token = entry
                        context.detach(tool_token)
                        tool_span.end()
                
                # Capture final message
                elif event.type == SessionEventType.ASSISTANT_MESSAGE and data:
                    if data.content:
                        print(f"Assistant: {data.content}")
        
        span.set_attribute("gen_ai.response.finish_reasons", ["stop"])
    
    except Exception as e:
        span.set_attribute("error.type", type(e).__name__)
        raise
    finally:
        # Clean up any unclosed tool spans
        for call_id, (tool_span, tool_token) in tool_spans.items():
            tool_span.set_attribute("error.type", "stream_aborted")
            context.detach(tool_token)
            tool_span.end()
        
        context.detach(token)
        span.end()

docs/opentelemetry-instrumentation.md:257

The import statement is missing the SessionConfig import. Based on the Python SDK's init.py, the correct import should include SessionConfig: from copilot import CopilotClient, SessionConfig. However, note that SessionConfig is a TypedDict and should be used as a dictionary literal rather than called as a constructor.

from copilot import CopilotClient, SessionConfig

docs/opentelemetry-instrumentation.md:79

The session.send() method returns a message ID string, not an async iterator of events. You cannot use "async for event in session.send(...)". Instead, events must be received by registering a handler with session.on(). The correct pattern is: session.on(lambda event: handle_event(event)) followed by await session.send({"prompt": "..."}). All examples using async for with session.send() are incorrect and need to be rewritten.

    async for event in session.send("Hello, world!"):
        # Process events and add attributes
        pass

docs/opentelemetry-instrumentation.md:77

The session.send() method expects a MessageOptions dictionary with a "prompt" key, not a plain string. According to the Python SDK implementation, it should be called as: session.send({"prompt": "Hello, world!"}). The current code would raise an error because strings don't have a get() method.

    async for event in session.send("Hello, world!"):

docs/opentelemetry-instrumentation.md:290

The CopilotClient constructor does not accept SessionConfig as a parameter. According to the Python SDK, CopilotClient should be initialized with optional CopilotClientOptions, and the SessionConfig (with model selection) should be passed to create_session() instead. Change this to: client = CopilotClient() followed by session = await client.create_session(SessionConfig(model="gpt-5")).

        client = CopilotClient(SessionConfig(model="gpt-5"))

docs/opentelemetry-instrumentation.md:236

The session.send() method expects a MessageOptions dictionary with a "prompt" key, not a plain string. This should be: session.send({"prompt": "Tell me a joke"}) instead of session.send("Tell me a joke").

async for event in session.send("Tell me a joke"):

Copilot · 2026-02-22T01:22:11Z