Skip to content

Conversation

@giles17
Copy link
Contributor

@giles17 giles17 commented Sep 16, 2025

Motivation and Context

This pull request introduces a new HostedImageGenerationTool, updates the response handling to properly format image data, and provides sample scripts and documentation for using these features.

  • Added the HostedImageGenerationTool class to agent_framework/_tools.py, enabling agents to generate images via AI services. This tool is now exported and available for use.
  • Integrated HostedImageGenerationTool into the OpenAI responses client (openai/_responses_client.py), allowing it to be recognized and included in agent toolsets.
  • Enhanced response content handling for image generation calls: image results are now converted to proper data URIs if needed, ensuring compatibility with downstream consumers and display utilities.
  • Added pillow as a dependency in pyproject.toml to support image display in the new examples.

Description

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

@Copilot Copilot AI review requested due to automatic review settings September 16, 2025 00:48
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation python labels Sep 16, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces image generation capabilities to the Agent Framework by adding a new HostedImageGenerationTool and enhancing the OpenAI responses client to support image generation workflows. It also includes comprehensive sample scripts demonstrating various agent capabilities.

  • Added HostedImageGenerationTool class to enable AI agents to generate images
  • Enhanced OpenAI responses client to properly handle image generation results with data URI formatting
  • Added pillow dependency and sample scripts for image generation, analysis, and structured output

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
python/packages/main/agent_framework/_tools.py Adds HostedImageGenerationTool class and exports it
python/packages/main/agent_framework/openai/_responses_client.py Integrates image generation tool support and enhances image result handling
python/pyproject.toml Adds pillow dependency for image processing
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_image_generation.py Sample script demonstrating image generation capabilities
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_image_analysis.py Sample script for image analysis using vision models
python/samples/getting_started/agents/openai_responses_client/openai_responses_client_with_structured_output.py Sample script showing structured output functionality
python/samples/getting_started/agents/openai_responses_client/README.md Updates documentation to include new sample scripts

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@github-actions github-actions bot changed the title OpenAI Responses Image Generation Tool + Added Samples Python: OpenAI Responses Image Generation Tool + Added Samples Sep 16, 2025
@markwallace-microsoft
Copy link
Member

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/main/agent_framework
   _tools.py3473889%166, 212, 255, 258–264, 386, 388, 390, 482–484, 529, 578, 609, 622–624, 643, 645–646, 676, 751–752, 834–835, 839, 855–857, 878–879, 887–888
packages/main/agent_framework/openai
   _responses_client.py3756782%140–141, 144–145, 151–152, 155, 162, 194, 203, 215, 245, 273–276, 278, 313–314, 360, 456, 471–472, 476–478, 526, 546–547, 560–561, 577–578, 611, 613, 637–638, 640–641, 643–644, 651–652, 665, 667, 740–746, 763–768, 787, 805, 815, 817, 835–836, 838–840
TOTAL7536128982% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
864 94 💤 0 ❌ 0 🔥 26.807s ⏱️

@stephentoub
Copy link
Member

cc: @ericstj

@giles17 giles17 enabled auto-merge September 19, 2025 17:27
@giles17 giles17 force-pushed the openai_samples_parity branch from 225ede0 to 3df2ee0 Compare September 19, 2025 18:47
@giles17 giles17 disabled auto-merge September 19, 2025 18:49
@giles17 giles17 closed this Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants