Stream text not working

[HuyThanh-09]

const result = streamText({
model: openai('gpt-4o-mini'),
system: 'You are a helpful assistant.',
messages: [
{
role: 'user',
content: [
{ type: 'text', text: messages },
{
type: "image_url",
image_url: {
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
},
},
],
},
],
});
Type '({ type: "text"; text: any; } | { type: "image_url"; image_url: { url: string; }; })[]' is not assignable to type 'string'.ts(2322)
index.d.ts(752, 5): The expected type comes from property 'content' which is declared here on type 'UIMessage'
I'm using OpenAi, if I change to
content: [
{ type: 'text', text: message },
{ type: 'image', image: url },
], as example in AI sdk it will not work

[brennanmceachran]

The error message you shared is a type error saying it should look like the below. The expected content format is like this:

[
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'image',
          image:
            'https://github.com/vercel/ai/blob/main/examples/ai-core/data/comic-cat.png?raw=true',
        },
      ],

I see you've tried this, but you mentioned it doesn't work when you tried it this way. Can you share that error when you tried it this way?

[brennanmceachran]

For me, using

streamText({
        model: openai('gpt-4o-mini'),
        messages: [
          {
            role: 'user',
            content: [
              { type: 'text', text: 'Describe the image in detail.' },
              {
                type: 'image',
                image:
                  'https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg',
              },
            ],
          },
        ],

Returns

The image depicts a serene landscape featuring a wooden boardwalk that meanders through a lush green field. The boardwalk is made of planks, slightly elevated above the ground, and appears to lead into the distance, inviting exploration.

On either side of the boardwalk, tall grasses sway gently in the breeze, creating a vibrant green carpet that stretches across the scene. The grass is interspersed with patches of wildflowers and shrubs, adding variety to the landscape.

In the background, a mix of trees and bushes can be seen, their foliage rich and healthy, suggesting a thriving ecosystem. The sky above is expansive, painted in soft shades of blue with wispy clouds scattered throughout, hinting at a pleasant day. The overall atmosphere is tranquil and inviting, perfect for a leisurely stroll in nature.

Are you using import { openai } from "@ai-sdk/openai" or open ai's sdk?

[HuyThanh-09]

I think the problem is that im using base64 instead of public url, do you know if this work for base64?

[HuyThanh-09]

i also got this Response {
status: 200,
statusText: '',
headers: Headers {
'Content-Type': 'text/plain; charset=utf-8',
'X-Vercel-AI-Data-Stream': 'v1'
},
body: ReadableStream { locked: false, state: 'readable', supportsBYOB: false },
bodyUsed: false,
ok: true,
redirected: false,
type: 'default',
url: ''
}

[HuyThanh-09]

also can i see ur full code, tsm

[brennanmceachran]

Better than my code would be this simplified example. (my code is littered with specific things for my app)

[lgrammel]

See https://sdk.vercel.ai/docs/foundations/prompts#image-parts for details on the image parts in prompts.