Annotations are not showing up at all

[DonaldNgai]

I just upgraded to 4.1 from 3.1 and it's been behaving super unexpectedly.

Here is what I am finding with the following code

Server:

return createDataStreamResponse({
        execute: async dataStream => {
            dataStream.writeData(`Hello`);

            const { textStream } = streamText({
                model: openai('gpt-4o'),
                messages,
                onChunk({ chunk }) {
                    console.log(chunk);
                    if ('textDelta' in chunk) {
                        dataStream.writeData(`d:${chunk.textDelta}`);
                    }
                    dataStream.writeMessageAnnotation({ chunk: '123' });

                },
                onFinish: ({ text, usage }) => {
                    dataStream.writeMessageAnnotation({
                        id: generateId(), // e.g. id from saved DB record
                        other: 'information',
                    });

                    dataStream.writeData('s:Finished');
                },
            });

            // Right now, streamText is broken as it just returns instantly which closes the stream. So this is the workaround.
            for await (const textPart of textStream) { }

            dataStream.writeData('s:Done');


        },
        onError: error => {
            // Error messages are masked by default for security reasons.
            // If you want to expose the error message to the client, you can do so here:
            console.log(error)
            return error instanceof Error ? error.message : String(error);
        },
    });

Client:

const {
    input,
    data,
    messages,
    setInput,
    handleSubmit,
    // isLoading
  } = useChat({
    api: "/api/ai/query",
    streamProtocol: 'data',
    body: {
      clientId: selectedClient.clientId,
      threadIds: threadIds
    },
  });

const handleQuery = async () => {
    if (selectedClient) {
      setLoading(true);
      setProgress(0);
      await handleSubmit();

      console.log(data)
    }
};

Even this simple code isn't working.

1. After the handleSubmit is called, the messages don't get updated ever. Regardless of any fix.
2. There are supposed to be annotations but with messages being empty, there is no information being passed back from the stream at all. Maybe fixing messages being empty might fix this but at the moment, This is completely broken
3. Data is empty UNLESS I have a hardcoded await after the streamText function on the server side as it's returning instantly now since 4.0 and so the stream is closed before anything can be passed back. This defeats the purpose of updating to not have streamText need an await anymore!
   4. import { MockLanguageModelV1 } from 'ai/test'; is now broken as well. When it is imported to use to help test, it now will have a runtime error as it still tries to import @ai-sdk/provider-utils/test which is not used in 4.0 anymore.

This is extremely frustrating as a developer to have a major update like this pretty much break everything.
If anyone has any insights, it'll be greatly appreciated. All of this is taken from the latest documentation. The code is taken from the examples given.

[lgrammel]

From which example is this taken?

Please check out https://sdk.vercel.ai/docs/ai-sdk-ui/streaming-data#sending-custom-data-from-the-server (in particular result.mergeIntoDataStream).

[DonaldNgai]

Hey Igrammel,

The example you gave is actually what I referenced for my code. I have tried the result.mergeIntoDataStream but that doesn't resolve any of the other issues.

1. Messages in useChat never gets updated
2. Since messages are empty, there are no annotations either.
3. I need to place this for await (const textPart of textStream) { } in order for it to not instantly return.
4. import { MockLanguageModelV1 } from 'ai/test'; includes provider-utils that breaks on runtime as it's not included in ^4.0 anymore.

[lgrammel]

4. makes me think you have not updated all @ai-sdk/* pkgs to latest and you see some errors because of that. Please ensure all packages are on the latest versions.

[DonaldNgai]

Thanks for the replies. I appreciate all the work you've put in.

I double check and you were right, some of the modules were not up to date. I updated it and 1-3 is not fixed.

4. still isn't working though. Here are my packages:

"ai": "^4.1.1",
"@ai-sdk/openai": "^1.1.1",
"@ai-sdk/provider-utils": "^2.1.1",
"@ai-sdk/react": "^1.1.1",

I get the error that the module can't be found:

Module not found: Can't resolve '_http_common'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/.pnpm/ai@4.1.1_react@18.3.1_zod@3.24.1/node_modules/ai/test/dist/index.mjs

Any thoughts?