Feat/ollama integration #194

csgulati09 · 2024-02-06T06:44:54Z

Title:

Provider: Ollama

Description: (optional)

Added Ollama as a provider
Covering 3 major features: Complete, chat and embed
Both, stream and non-stream functions are supported
Add support for custom_host setting in config and alss x-portkey-custom-host header.
Add support for forward_headers setting in config and also x-portkey-forward-headers headers.

Motivation: (optional)

This allows the users to use their self hosted Ollama via Portkey gateway.
To allow adding custom base url to use proxy instead of directly using provider base_url.
To allow users to add custom headers to LLM request without any overwrite from gateway.

Related Issues: (optional)

…and google providers

…y, removed completion route

src/providers/ollama/chatComplete.ts

VisargD · 2024-02-14T09:46:20Z

src/providers/ollama/api.ts

+  embed:"/api/embeddings",
+  getEndpoint: (fn: string, API_VERSION: string, url?: string) => {
+    let mappedFn = fn;
+    if (fn === "proxy" && url && url?.indexOf("/chat/completions") > -1) {


This is not required as /chat/completions will always point to the unified route.

src/providers/ollama/api.ts

src/providers/ollama/chatComplete.ts

src/utils.ts

src/handlers/streamHandler.ts

… from stream handler, default split pattern

…ard-headers

…headers Feat/custom host and forward headers

Chandeep added 5 commits January 31, 2024 19:08

completion without stream in progress

3582c17

ollama completion with and without stream integrated

c8bad18

ollama chat with and without stream integrated + small fix in cohere …

4932faf

…and google providers

Ollama embed integration + ASK: baseUrl for Ollama

cf8f223

proxy path URL handled

139b8da

csgulati09 requested a review from VisargD February 6, 2024 06:44

csgulati09 self-assigned this Feb 6, 2024

vrushankportkey mentioned this pull request Feb 7, 2024

[Bug]: portkey_ai.api_resources.exceptions.NotFoundError: Error code: 404 - Unable to find requested model. Please confirm that the model exists and you have permission. (Request ID: ) Portkey-AI/portkey-python-sdk#84

Closed

1 task

csgulati09 added 3 commits February 13, 2024 19:03

feat: chatCompletion from openai compatible, embedding from native wa…

2b235b6

…y, removed completion route

chore: merged main on feat/ollama-integration

25933c4

merged main on feat/ollama-integration

6c87e25