Quickstart Installation ????

[maddes8cht]

I don't get it.
I know how to run llama.cpp main.exe with my models under windows.
In this respect I get as far as testing llama.cpp installation with Quickstart Installation.
My problem starts at the Running gpt-llama.cpp section.
Because there is nowhere here where I actually specify my model, or where my llama.cpp is, or the main.exe or anything else from llama.cpp.
How should gpt-llama.cpp know that somewhere a main.exe is running?
Does the main.exe have to run with a model somewhere? That is not stated anywhere either.

I have no idea what to do with this Quickstart Installation.

Does anyone understand it?
Does anyone know what to do here?
I'm really confused.

[keldenl]

gpt-llama.cpp infers all this information with your linked mode. It relies on the fact that your model is in your llama.cpp folder, and detects your main.exe file from that path llama.cpp\models\path\to\model.bin (takes llama.cpp\ and appends llama.cpp\main.exe).

You just need to set your OPENAI-API_KEY to your model path llama.cpp\models\path\to\model.bin and gpt-llama.cpp will do the rest for you :)

[maddes8cht]

So there needs to be Llama.cpp's main.exe in the same directory as my model?
Is that enough to make it work?
Then point to the models path, and that's it?

[maddes8cht]

Finally got it to work.

I think there is a problem with the documentation.
Interestingly, you can work through the Quickstart and it runs - especially if you have no idea what you're doing, or if you know exactly what it is anyway.
I found myself thinking at several points: surely something is missing here? This can't work?
I think that's because the concept doesn't become clear until you actually have it running.

Whereas it hasn't become clear to me yet what the function of the swagger app is, but you don't need it anyway.

As far as I understand it now, it is that you get the server running without loading any instance of Llama.cpp, without a model, without anything working.
The model is loaded only on request of the using application, which tells the location of the model. This assumes a directory structure that requires a Llama.cpp main in the right place.
If I have understood the embedding feature correctly, then one does not need also this presupposed directory structure any longer.

This may all sound obvious to the authors, but the lack of this info can drive a semi-literate prospective user to despair.

So if someone confirms that my description is correct, I would suggest to add some paragraphs to the Quickstart installation.
I am happy to help with the wording.
I think it would help new users a lot.