OpenAI Web Search Image Results: Grounding Responses in Current Visuals

OpenAI expanded the web search tool in the Responses API on June 9, 2026. In addition to text results, web search can now return image results — enabling agents to ground responses in current web visuals without requiring a separate image search integration. Part of our Builder’s Log.

What Changed

The built-in web search tool previously returned only text results: links, titles, snippets, and source metadata. As of June 9, image results are available as a first-class output type. The agent can search for and return image URLs from the web, directly via the same tool call that retrieves text.

This matters for use cases where the answer is visual: product photos, event imagery, place photos, technical diagrams, and anything where current web images add context that text cannot.

Enabling Image Results

Add search_content_types to the web search tool definition:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.5",
    input=[{"role": "user", "content": "Show me what the new iPhone 17 Ultra looks like."}],
    tools=[{
        "type": "web_search",
        "search_content_types": ["image", "text"],
        "image_settings": {
            "max_results": 5,
            "caption": True,
        },
    }],
)

search_content_types accepts "text", "image", or both, as documented in OpenAI’s web search guide.

Response Structure

Image results appear in the web_search_call item, separate from the assistant message, as documented by OpenAI. Parse the results from the tool call response, not from the model’s text output:

for item in response.output:
    if item.type == "web_search_call":
        for result in item.results:
            if result.type == "image_result":
                print("Image URL:", result.image_url)
                print("Source page:", result.source_website_url)
                print("Thumbnail:", result.thumbnail_url)
                print("Caption:", result.caption)

Each image_result object contains the following documented fields:

thumbnail_url and caption are optional — they’ll be None when the source page doesn’t provide them. Check before using.

What the Model Does With Image Results

The model receives the image metadata — image_url, source_website_url, thumbnail_url, and caption — as structured content in the tool result. It can:

• Reference the image URLs in its response text
• Describe what the images show (based on the caption field)
• Select which images are most relevant to surface to the user
• Incorporate image URLs into markdown or HTML output

OpenAI’s documented image_result schema contains only URL and text metadata fields — no image bytes or visual embedding. So a result by itself doesn’t give the model pixel-level access to the image. To have the model actually see and interpret the image content, pass the URL back as a vision input in a follow-up turn using the Responses API’s input_image content type, as shown below.

image_settings Options

"image_settings": {
    "max_results": 10,    # Maximum number of image results to return (default unspecified)
    "caption": True,      # Request caption text alongside image URLs
}

Both fields are optional, per OpenAI’s image_settings documentation: max_results sets how many image results to request, and caption requests short image descriptions when available. When caption is not set to True, the caption field on results will be None even when the source page provides one.

Display Patterns

Inline image grid in a chat interface:

image_results = [
    r for item in response.output
    if item.type == "web_search_call"
    for r in item.results
    if r.type == "image_result"
]

html_images = "".join(
    f'<a href="{r.source_website_url}"><img src="{r.thumbnail_url or r.image_url}" '
    f'alt="{r.caption or ""}"></a>'
    for r in image_results
    if r.image_url
)

Passing image URLs back to the model for vision analysis (using the Responses API’s input_image content type, not the Chat Completions image_url-object shape):

# After retrieving image URLs, pass them back for visual interpretation
followup = client.responses.create(
    model="gpt-5.5",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Based on these search results, describe what you see."},
                *[
                    {"type": "input_image", "image_url": r.image_url}
                    for r in image_results[:3]  # limit to 3 for context budget
                    if r.image_url
                ],
            ],
        }
    ],
)

Use Cases

Product research agents: Retrieve current product photos alongside reviews and pricing — useful when catalog images are outdated or product isn’t in your own database.

Travel and places: Ground destination recommendations with photos of landmarks, hotels, or neighborhoods, sourced from the current web rather than static assets.

News and events: Surface images from recent news coverage to accompany text summaries, giving users a visual anchor to breaking events.

Technical documentation: When answering questions about physical hardware, interface screenshots, or UI states that change over time, retrieve current screenshots rather than relying on static documentation images.

What Image Search Does Not Provide

• Image content analysis during search: OpenAI’s documented image_result fields — image_url, source_website_url, thumbnail_url, caption — are URL and text metadata only. No pixel data is included, so the model doesn’t “see” the retrieved images while composing the initial response.
• License information: OpenAI’s documented image_result schema does not include a licensing or rights field. Responsibility for rights review is on the caller.

Combining Image and Text Results

Image and text results use two different, documented mechanisms in the response, per OpenAI’s web search guide: image results appear as image_result objects in the web_search_call item’s results array, while text citations appear as url_citation annotations attached to the assistant’s output_text message content — the same annotation mechanism the tool used for text-only results before this update. When both content types are requested, expect both mechanisms in the same response:

for item in response.output:
    if item.type == "web_search_call":
        image_results = [r for r in item.results if r.type == "image_result"]
        print(f"Image results: {len(image_results)}")

    if item.type == "message":
        citations = [
            a
            for c in item.content
            for a in c.get("annotations", [])
            if a.get("type") == "url_citation"
        ]
        print(f"Text citations: {len(citations)}")