feature/fallback providers

[sonic182]

Closes #34

---

Implement multi-provider support with provider routing and failover

• Introduce a new :providers list in LlmComposer.Settings to replace deprecated :provider and :provider_opts keys.
• Add validation in LlmComposer to enforce/suggest exclusive use of :providers and warn about deprecated keys.
• Implement LlmComposer.ProviderRunner to handle provider execution, supporting multiple providers with fallback logic.
• Add LlmComposer.ProviderRouter behaviour to define routing strategies for provider selection, failure handling, and blocking.
• Provide a simple default provider router LlmComposer.ProviderRouter.Simple with exponential backoff blocking on provider failures.
• Refactor LlmComposer.run_completion/3 to delegate to the ProviderRunner for provider selection and execution.
• Optimize LlmComposer.Cache.Ets by switching put and delete calls to asynchronous casts for improved performance.
• Maintain backward compatibility with deprecated settings keys, issuing warnings and supporting legacy calls until version 0.11.0.
• Changed response_format key for response_schema for better definition of structured outputs that works for more than one provider, for the same schema definition

---

You may wanna test the router simple.ex that has the backoff blocking of provider, here is a very basic example, make sure to have ollama service stopped or not installed

openaikey = File.read!("./openaikey") |> String.trim()
Application.put_env(:llm_composer, :openai_key, openaikey)

defmodule MyCustomChat do

  @settings %LlmComposer.Settings{
    providers: [
      # Ollama not started, so will cause errors
      {LlmComposer.Providers.Ollama, [model: "whatever_its_gonna_be_not_started"]},
      {LlmComposer.Providers.OpenAI, [model: "gpt-4o-mini"]}
    ],
    system_prompt: "You are an assistant specialized in history.",
    auto_exec_functions: false,
    functions: []
  }

  def run_custom_chat() do
    # Define a conversation history with user and assistant messages
    messages = [
      %LlmComposer.Message{type: :user, content: "how mutch is 1 + 1?"},
    ]
    
    {:ok, res} = LlmComposer.run_completion(@settings, messages)
    
    res.main_response
  end
end


{:ok, pid} = LlmComposer.ProviderRouter.Simple.start_link([])

Enum.each(0..7, fn _ ->
  IO.inspect(MyCustomChat.run_custom_chat())
end)

place the example in llm_composer folder and do mix run <the example>.ex

[sonic182]

moved

[mmacia]

I think that there's a lot of abstraction leaks launched to the final user (cache, router, runner, etc.) that are hidden behind default values. Let's try it and see if it works for us.

[victor23k]

Really nice features! Next time I would split some of the smaller changes into different PRs.