From 7c66c4bdb97a72e8c9fcc39ab1509a8059f7caa1 Mon Sep 17 00:00:00 2001 From: Dan Sun Date: Tue, 24 Dec 2024 22:31:49 -0500 Subject: [PATCH] Update version to 0.14.1 and runtime doc (#435) * Update version to 0.14.1 and runtime doc Signed-off-by: Dan Sun * Fix wording Signed-off-by: Dan Sun --------- Signed-off-by: Dan Sun --- docs/modelserving/v1beta1/serving_runtime.md | 11 ++++------- mkdocs.yml | 2 +- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/docs/modelserving/v1beta1/serving_runtime.md b/docs/modelserving/v1beta1/serving_runtime.md index 6f5f5c962..f90507774 100644 --- a/docs/modelserving/v1beta1/serving_runtime.md +++ b/docs/modelserving/v1beta1/serving_runtime.md @@ -2,8 +2,8 @@ KServe provides a simple Kubernetes CRD to enable deploying single or multiple trained models onto model serving runtimes such as [TFServing](https://www.tensorflow.org/tfx/guide/serving), [TorchServe](https://pytorch.org/serve/server.html), [Triton Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs). -In addition [ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) is the Python model serving runtime implemented in KServe itself with prediction v1 protocol, -[MLServer](https://github.com/SeldonIO/MLServer) implements the [prediction v2 protocol](https://github.com/kserve/kserve/tree/master/docs/predict-api/v2) with both REST and gRPC. +For Hugging Face models, KServe provides [Hugging Face Server](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) for hosting the transformer based models with Open Inference and OpenAI Protocol. +In addition [ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) is the Python model serving runtime implemented in KServe itself with prediction v1 and Open Inference Protocol(v2), These model serving runtimes are able to provide out-of-the-box model serving, but you could also choose to build your own model server for more complex use case. KServe provides basic API primitives to allow you easily build custom model serving runtime, you can use other tools like [BentoML](https://docs.bentoml.org/en/latest) to build your custom model serving image. @@ -36,19 +36,16 @@ In a given serving runtime directory the pyproject.toml file contains the exact | Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples | |-------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------| | [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model/README.md) | -| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.5.0 (MLServer) | 4 | [LightGBM Iris V2](./lightgbm/README.md) | | [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 4 | [LightGBM Iris](./lightgbm/README.md) | | [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.5.0 (MLServer) | 2 | [MLFLow wine-classifier](./mlflow/v2/README.md) | | [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml/README.md) | -| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.5.0 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2/README.md) | | [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 1.5 | [SKLearn Iris](./sklearn/v2/README.md) | | [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow/README.md) | | [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.9.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve/README.md) | | [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript/README.md) | -| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.5.0 (MLServer) | 2 | [XGBoost Iris V2](./xgboost/README.md) | | [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 2 | [XGBoost Iris](./xgboost/README.md) | -| [HuggingFace ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.39.2/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v1, v2 | -- | v{{ kserve_release_version }} (KServe) | 4 ([Transformers](https://pypi.org/project/transformers/4.43.4/)) | -- | -| [HuggingFace VLLM ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.43.4/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v2 | -- | v{{ kserve_release_version }} (KServe) | 0 ([Vllm](https://pypi.org/project/vllm/0.6.1.post2/)) | -- | +| [HuggingFace ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.39.2/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v1, v2, OpenAI | -- | v{{ kserve_release_version }} (KServe) | 4 ([Transformers](https://pypi.org/project/transformers/4.43.4/)) | -- | +| [HuggingFace VLLM ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.43.4/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v2, OpenAI | -- | v{{ kserve_release_version }} (KServe) | 0 ([Vllm](https://pypi.org/project/vllm/0.6.1.post2/)) | -- | diff --git a/mkdocs.yml b/mkdocs.yml index f82030c64..346b6c88d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -200,7 +200,7 @@ plugins: extra: - kserve_release_version: 0.14.0 + kserve_release_version: 0.14.1 social: - icon: fontawesome/brands/github link: https://github.com/kserve