Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow python packages to contribute to LlamaChatCompletionHandlerRegistry #1715

Open
axel7083 opened this issue Aug 29, 2024 · 1 comment · May be fixed by #1716
Open

Allow python packages to contribute to LlamaChatCompletionHandlerRegistry #1715

axel7083 opened this issue Aug 29, 2024 · 1 comment · May be fixed by #1716

Comments

@axel7083
Copy link

Is your feature request related to a problem? Please describe.

Today, the [llama-cpp/llama_chat_format.py] contains 25 chat format, and 4 chat_completion_handler, this currently force the different actors to contribute to this never ending growing file.

This is the case for the functionary models, which has to keep updating the handlers to support their newer models.

This process can be slower than their pace of release since they have to get approval on this repository, the amazing people behind the functionary models have a repository with the necessary code to transform the generated content to proper CreateChatCompletionStreamResponse, and it would make sense that this would be their responsibility.

Describe the solution you'd like

python (>3.3) offers a lot of way to load code from other packages, or packages to contribute to a main packages. This would have a lot of advantages, as model provider could maintain their own packages, and rely on their own testing/versioning.

Additional context
Add any other context or screenshots about the feature request here.

@axel7083 axel7083 changed the title Allow python packages to contribute to LlamaChatCompletionHandler Allow python packages to contribute to LlamaChatCompletionHandlerRegistry Aug 29, 2024
@axel7083
Copy link
Author

axel7083 commented Sep 5, 2024

Hey @abetlen ! Do you have any opinion on this matter ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant