Handle arbitrary length StreamChunkPart

[mattrossman]

Type

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

This changes OpenAiClient.StreamChunk.text and OpenAiClient.StreamChunk.asAiResponse getters to respect arbitrary length this.parts instead of only considering the first part.

After patching this locally, I was able to use Google's Gemini models via their OpenAI compatibility API.

Areas to Review:

I'm not sure what is expected when there's multiple "Content" parts, all I know is that Gemini delivers one "Usage" and one "Content" part in the same chunk. For now I've .join()-ed all content parts into a single string.

I'm not certain why the asAiReponse implementation produces an empty string of content for the case when no parts are present (versus just providing an empty parts array).

effect/packages/ai/openai/src/OpenAiClient.ts

Lines 293 to 298 in 9a88bb9

	if (this.parts.length === 0) {
	return AiResponse.AiResponse.fromText({
	role: AiRole.model,
	content: ""
	})
	}

I've preserved this behavior in case, though I imagine it could be removed.

Related

• Closes @effect/ai-openai discards meaningful content with multiple StreamChunkPart #4629

[changeset-bot]

🦋 Changeset detected

Latest commit: 4513ae8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@effect/ai-openai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[IMax153]

I agree that we should leave this logic in here for now, at least until I've gotten some test cases written for AiResponse.