How does Funcchain handle different prompt templates?

[ahuang11]

Hi again. I'm wondering, does funcchain handle different chat templates internally?

e.g. for Llama

<s>[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible.
<</SYS>>

 [/INST]

vs Mistral

text = "<s>[INST] What is your favourite condiment? [/INST]"
"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
"[INST] Do you have mayonnaise recipes? [/INST]"

References:
https://www.reddit.com/r/LocalLLaMA/comments/1afweyw/comment/kofabzx/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
https://www.promptingguide.ai/models/mistral-7b#chat-template-for-mistral-7b-instruct

[shroominic]

Its just using the langchain templating so langchain_community is doing all the work converting every model provider.

[ahuang11]

I don't think so; I think funcchain is hardcoding prompt formats.
https://github.com/shroominic/funcchain/blob/main/src/funcchain/model/patches/llamacpp.py#L269-L283

Llama-cpp-python actually supports different chat_formats

https://github.com/abetlen/llama-cpp-python/blob/b1637c2319936df0ecf1b3eb18ca971b346a147e/llama_cpp/llama_chat_format.py#L609

[shroominic]

yess true we discussed in discord need to look into that!