Support more than one model

[lingster]

Given the range of models that ollama can run and given that we now have smaller models that are great for specific tasks, how hard would it be for specific agents to run with a specific model. Example Qwen-coder for coding tasks, Qwen-vl for image related tasks or perhaps llama 3.3 for overall task management.

[ErikBjare]

It would be pretty easy to do such routing here:

gptme/gptme/llm/__init__.py

Lines 50 to 63 in 803d6e5

	def reply(
	messages: list[Message],
	model: str,
	stream: bool = False,
	tools: list[ToolSpec] | None = None,
	) -> Message:
	if stream:
	return _reply_stream(messages, model, tools)
	else:
	print(f"{PROMPT_ASSISTANT}: Thinking...", end="\r")
	response = _chat_complete(messages, model, tools)
	print(" " * shutil.get_terminal_size().columns, end="\r")
	print(f"{PROMPT_ASSISTANT}: {response}")
	return Message("assistant", response)

I'm not super interested in it though, but should be easy for gptme to modify itself to do!

[0xbrayo]

@ErikBjare This was implemented in the last release? using /model

[ErikBjare]

Yes, could also be done by adding tools to outsource such reasoning: #416

We are not doing any auto-routing, but I don't think we will be, at least not for now.