-
Notifications
You must be signed in to change notification settings - Fork 565
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FEAT: Add support for Flexible Model #1671
Conversation
@yiboyasss Could you review the frontend part of this PR? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left some comments
Co-authored-by: Xuye (Chris) Qin <qinxuye@gmail.com>
When registering a model, there should not be two models with the same name, an error message should be given! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for your contribution.
Flexible Model is a customizable inference pipeline within xinference, supporting a range of tasks including image segmentation, traditional machine learning regression, and more.
Mechanism:
Create the model object with custom definitions and initialization functions.
Use xinference.model.flexible.launchers.transformers to load transformer models.
Invoke the model via the /v1/flexible/infers endpoint.
Access the model using the SDK’s infer interface.
Examples: