-
Notifications
You must be signed in to change notification settings - Fork 587
Python: OpenAI Responses Image Generation Tool + Added Samples #763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces image generation capabilities to the Agent Framework by adding a new HostedImageGenerationTool and enhancing the OpenAI responses client to support image generation workflows. It also includes comprehensive sample scripts demonstrating various agent capabilities.
- Added
HostedImageGenerationToolclass to enable AI agents to generate images - Enhanced OpenAI responses client to properly handle image generation results with data URI formatting
- Added
pillowdependency and sample scripts for image generation, analysis, and structured output
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
python/packages/main/agent_framework/_tools.py |
Adds HostedImageGenerationTool class and exports it |
python/packages/main/agent_framework/openai/_responses_client.py |
Integrates image generation tool support and enhances image result handling |
python/pyproject.toml |
Adds pillow dependency for image processing |
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_image_generation.py |
Sample script demonstrating image generation capabilities |
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_image_analysis.py |
Sample script for image analysis using vision models |
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py |
Sample script showing structured output functionality |
python/samples/getting_started/agents/openai_responses_client/README.md |
Updates documentation to include new sample scripts |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
...s/getting_started/agents/openai_responses_client/openai_responses_client_image_generation.py
Outdated
Show resolved
Hide resolved
Python Test Coverage Report •
Python Unit Test Overview
|
||||||||||||||||||||||||||||||||||||||||
...les/getting_started/agents/openai_responses_client/openai_responses_client_image_analysis.py
Outdated
Show resolved
Hide resolved
...s/getting_started/agents/openai_responses_client/openai_responses_client_image_generation.py
Outdated
Show resolved
Hide resolved
...ing_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py
Outdated
Show resolved
Hide resolved
...ing_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py
Outdated
Show resolved
Hide resolved
python/packages/main/agent_framework/openai/_responses_client.py
Outdated
Show resolved
Hide resolved
...ing_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py
Outdated
Show resolved
Hide resolved
...ing_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py
Outdated
Show resolved
Hide resolved
|
cc: @ericstj |
225ede0 to
3df2ee0
Compare
Motivation and Context
This pull request introduces a new
HostedImageGenerationTool, updates the response handling to properly format image data, and provides sample scripts and documentation for using these features.HostedImageGenerationToolclass toagent_framework/_tools.py, enabling agents to generate images via AI services. This tool is now exported and available for use.HostedImageGenerationToolinto the OpenAI responses client (openai/_responses_client.py), allowing it to be recognized and included in agent toolsets.pillowas a dependency inpyproject.tomlto support image display in the new examples.Description
Contribution Checklist