Vercel ai sdk integration

[PaperBoardOfficial]

LLM Provider Integration Changes

Important considerations

• llm.ts: Created a single class to handle the logic for llm calls.
• Json Schema: The providers added support structured output json schema.
• Temperature: No need to take care of parameters that are not suported by the model because Vercel AI SDK takes care of it. For example, here, if the model is a reasoning model, vercel will not send the temperature to it as the model doesn't support it.
• Context Window length: Not implemented. I'll raise another ticket for it.
• Note: I have only tested it for Gemini. Need to test it for other models.

Core Architecture Changes

New Files Added

• model-provider.ts

  • Manages different LLM providers (OpenAI, Anthropic, Google Gemini, Mistral, xAI, DeepSeek)
  • Handles model selection and configuration
  • Provides getModel() and getSchemaModel() methods

• llm.ts

  • Implements Singleton pattern for LLM client
  • Uses Vercel AI SDK's generateText and generateObject functions
  • Provides type-safe responses using Zod schemas
  • Methods:
    • getText(): For raw text generation
    • getObject(): For structured output with schema validation

Core Utilities Updates

Transform Module (transform.ts)

• Replaced OpenAI with LLMClient
• Added Zod schema validation for mapping generation
• Uses generateObject for JSONata expression generation

Schema Module (schema.ts)

• Migrated to LLMClient for schema generation
• Uses generateText for JSON schema generation
• Uses ModelProvider for schema model selection

API Module (api.ts)

• Updated to use LLMClient for API configuration

Extract Module (extract.ts)

• Integrated LLMClient for extraction configuration
• Added Zod schema validation

Test Suite Updates

Modified Test Files

• transform.test.ts

  // Replaced OpenAI mocks with LLMClient
  const mockGetObject = vi.fn();
  vi.mocked(LLMClient).getInstance = vi.fn(() => ({
    getText: mockGetText,
    getObject: mockGetObject
  }));

• schema.test.ts

  // Added ModelProvider mock
  vi.mock('./model-provider.js', () => ({
    default: {
      getSchemaModel: vi.fn(() => 'gpt-4o')
    }
  }));

• api.test.ts and extract.test.ts

  • Removed OpenAI-specific mocks
  • Added LLMClient mocking
  • Updated test assertions for new architecture

Package Dependencies

Added Vercel AI SDK Dependencies

{
  "dependencies": {
    "ai": "^4.1.54",
    "@ai-sdk/anthropic": "^1.1.14",
    "@ai-sdk/deepseek": "^0.1.12",
    "@ai-sdk/google": "^1.1.19",
    "@ai-sdk/mistral": "^1.1.13",
    "@ai-sdk/openai": "^1.2.0",
    "@ai-sdk/xai": "^1.1.12",
  }
}

Configuration Updates

Environment Variables

• Added provider selection:

  LLM_PROVIDER=openai|google|anthropic|mistral|deepseek|xai
  SCHEMA_GENERATION_PROVIDER=openai|google|anthropic|mistral|deepseek|xai

• Added model selection:

  LLM_MODEL=<provider-specific-model>
  SCHEMA_GENERATION_MODEL=<provider-specific-model>

• Added provider API keys:

  GOOGLE_GENERATIVE_AI_API_KEY=
  OPENAI_API_KEY=
  ANTHROPIC_API_KEY=
  MISTRAL_API_KEY=
  DEEPSEEK_API_KEY=
  XAI_API_KEY=

Documentation Updates

• Updated architecture.md documentation
• Updated self-hosting,md documentation
• Updated introduction.mdx documentation

[PaperBoardOfficial]

@sfaist This is ready for review. 🤠
Please let me know if there's something you feel can be improved I really had to read docs of vercel ai sdk (and litellm before, because cursor sucks at latest ai related libraries), so I want that this PR gets merged.
Note: the openai key was not working, so I didn't test it. I only tested it for gemini.
Also, feel free to assign some more issues to me.

[stefanfaistenauer]

Thanks a lot for this PR. Looks great. I will take some more time tomorrow to review and kick off the merging process. Since this has implications for our hosted version we will have to be careful and take some time to finish that. Will do that asap.