Proposal to Stop In-Tree Provider Implementations

[leseb]

Background:

With the upcoming ability to Bring Your Own Provider (BYOP) in #1672, llama-stack is evolving to offer a more modular and extensible approach to provider integrations. This new capability enables users to develop, deploy, and maintain their own providers outside the core repository, reducing the burden on the main codebase and improving maintainability. This is a common practice and major softwares like OpenStack and Kubernetes use a similar mechanism.

Proposal:

To streamline our architecture and encourage adoption of the BYOP model and put the focus on the API, we propose to stop accepting any new provider implementations in-tree. This change will:

• Reduce maintenance overhead for core maintainers.
• Encourage provider developers to leverage the BYOP framework, fostering a more scalable ecosystem.
• Improve system modularity by decoupling provider implementations from the core API.
• Minimize the risk of bloating the codebase with providers that may not be widely used or actively maintained which is something hard to track.
• Minimize the impact of introducing a new API or modifying an existing one, ensuring that changes are localized rather than affecting the entire codebase.

Action Plan:

• Announce the deprecation of in-tree provider implementations and communicate the transition plan to the community during the weekly community call
• Provide documentation and tooling to support BYOP adoption - will be done as part of feat: ability to execute external providers #1672 or has a followup.
• Establish a clear policy for ongoing support of existing in-tree providers while transitioning to an external model - TBD
• Stop accepting any new Provider in-tree
• Initiate the removal of some providers with a deprecation policy