Adding ollama api connection

[ZaMeR12]

Hi, i want to do my own chat AI interface for study and personnal project. I find the library interesting, the only problem is that I use Ollama as LLM provider (local server for llms). So Ithink that can be interesting to add Ollama's API to the list of the AI connection possible. That will help people like me to have a framework for it like this one instead to deal with multiple libraries. (ui chat components, axios, material ui,etc...) I understand that is not something to help most people, but certainely be interesting to think about it at least.

Thanks you for your consideration.

[ZaMeR12]

Finally i found that Ollama offer a openapi compatibillity, but this is an experimental function made by Ollama's dev. So it's still not the best option. So i still think that will be a good idea to had a connection for the Ollama initial api.

[keesfluitman]

Hey @ZaMeR12
Im also looking into this. I was tryingto do it in sveltekit and got a response, but cant get it to stream in chat yet. Im too inexperienced in web dev but ill try it in flask. More capable in flask atm.

https://ollama.com/blog/openai-compatibility

U wanna pm and try to set it up?

[OvidijusParsiunas]

Hey folks,

Sorry for no response for so long as I currently barely have any free time after work as I just moved to a new country. I am committed to expanding the directConnection APIs that we offer to our users and will explore Ollama in the near future.

Currently, I would advise you to use the connection handler property which allows you to connect to any API out there, so if you have your own server or are connecting to an existing server, I would highly recommend checking it out.

You can also alternatively explore the use of the interceptor functions which allow you to modify the outgoing and incoming messages.

Apologies for the lack of recent developments and I will try to tackle these issues as soon as I get a chance!

Thankyou.

[keesfluitman]

Hey folks,

Sorry for no response for so long as I currently barely have any free time after work as I just moved to a new country. I am committed to expanding the directConnection APIs that we offer to our users and will explore Ollama in the near future.

Currently, I would advise you to use the connection handler property which allows you to connect to any API out there, so if you have your own server or are connecting to an existing server, I would highly recommend checking it out.

You can also alternatively explore the use of the interceptor functions which allow you to modify the outgoing and incoming messages.

Apologies for the lack of recent developments and I will try to tackle these issues as soon as I get a chance!

Thankyou.

Hey don't worry man. Great free stuff to learn for me! As a little joke Im gonna use it to create a chat for my soccer team, where the parents can ask when the next tournament is and other info (as some parents just never read my website and always call,text or ask privately). Just for a few days, as it will cost me too much power to keep the PC running all the time.

But it wasn't too hard to set it up. In Sveltekit I store the soccerData, retrieve it in page.svelte, have it send with the body to the chat endpoint. Have that extract the data and send it with an initial system prompt, so when a user starts chatting, it's User specific data (so his/her children, matches etc.) is loaded in the llms chat history. With llama3.2 that works perfectly fine.

Even though im still a noob at this javascript stuff though. Never worked with frameworks in javascript, only some flask. Do you also give private lessons by any chance? Or too busy with work? Ill donate for sure, and Im willing to pay a fair price for regular efficient lessons. Or do you know a good place to find help?

[ZaMeR12]

Hey folks,

Sorry for no response for so long as I currently barely have any free time after work as I just moved to a new country. I am committed to expanding the directConnection APIs that we offer to our users and will explore Ollama in the near future.

Currently, I would advise you to use the connection handler property which allows you to connect to any API out there, so if you have your own server or are connecting to an existing server, I would highly recommend checking it out.

You can also alternatively explore the use of the interceptor functions which allow you to modify the outgoing and incoming messages.

Apologies for the lack of recent developments and I will try to tackle these issues as soon as I get a chance!

Thankyou.

No worry, it's just I wanted to use this library to make my codingvtime easuer and faster. So using the alternative wasn't the good idea for my student project. Also with all the project i made in class, i am able to do it kinda globally myself. So like i said that was more to easier my life for what i wanted than other things.

So if i have a simillar project in the future and see that the new versions fit better for me i will use it that time. I mean i have already built my base for my current project so it's will be complex to switch to your library right now.

I wiah you good luck.

[ZaMeR12]

Hey folks,
Sorry for no response for so long as I currently barely have any free time after work as I just moved to a new country. I am committed to expanding the directConnection APIs that we offer to our users and will explore Ollama in the near future.
Currently, I would advise you to use the connection handler property which allows you to connect to any API out there, so if you have your own server or are connecting to an existing server, I would highly recommend checking it out.
You can also alternatively explore the use of the interceptor functions which allow you to modify the outgoing and incoming messages.
Apologies for the lack of recent developments and I will try to tackle these issues as soon as I get a chance!
Thankyou.

Hey don't worry man. Great free stuff to learn for me! As a little joke Im gonna use it to create a chat for my soccer team, where the parents can ask when the next tournament is and other info (as some parents just never read my website and always call,text or ask privately). Just for a few days, as it will cost me too much power to keep the PC running all the time.

But it wasn't too hard to set it up. In Sveltekit I store the soccerData, retrieve it in page.svelte, have it send with the body to the chat endpoint. Have that extract the data and send it with an initial system prompt, so when a user starts chatting, it's User specific data (so his/her children, matches etc.) is loaded in the llms chat history. With llama3.2 that works perfectly fine.

Even though im still a noob at this javascript stuff though. Never worked with frameworks in javascript, only some flask. Do you also give private lessons by any chance? Or too busy with work? Ill donate for sure, and Im willing to pay a fair price for regular efficient lessons. Or do you know a good place to find help?

Hi @keesfluitman , so you want to learn more about javascript and framework with nodejs? I am right now in the end of my computer sciences study (4 years). So i have learn a lot about OOP (object oriented programming) and differents coding language. I learnt to use ExpressJS (back end/api framework) and React (front end/graphical interface framework) in Typescript (typed version of javascript). I also know different type of databases: some oriented logical (MySQL,SQLite) and some oriented document database like MongoDB. So if you are interested, go to my github profile and contact me with my links on here. In there we will figure out if we can do something to make you learn some of these stuff.