Skip to content
@bentoml

BentoML

Build fast and reliable model serving systems

Welcome to BentoML 👋 Twitter Follow Slack

github banner

What's cooking? 👩‍🍳

🍱 BentoML: The Unified Serving Framework for AI/ML Systems

BentoML is a Python library for building online serving systems optimized for AI apps and model inference. It supports serving any model format/runtime and custom Python code, offering the key primitives for serving optimizations, task queues, batching, multi-model chains, distributed orchestration, and multi-GPU serving.

🎨 Examples: Learn by doing!

A collection of examples for BentoML, from deploying OpenAI-compatible LLM service, to building voice phone calling agents and RAG applications. Use these examples to learn how to use BentoML and build your own solutions.

🦾 OpenLLM: Self-hosting Large Language Models Made Easy

Run any open-source LLMs (Llama, Mistral, Qwen, Phi and more) or custom fine-tuned models as OpenAI-compatible APIs with a single command. It features a built-in chat UI, state-of-the-art inference performance, and a simplified workflow for production-grade cloud deployment.

☁️ BentoCloud: Unified Inference Platform for any model, on any cloud

BentoCloud is the easist way to build and deploy with BentoML, in our cloud or yours. It brings fast and scalable inference infrastructure into any cloud, allowing AI teams to move 10x faster in building AI applications with ML/AI models, while reducing compute cost - by maxmizing compute utilization, fast GPU autoscaling, minimimal coldstarts and full observability. Sign up today!.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Python 7.9k 859

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    Python 11.6k 754

  3. Yatai Yatai Public

    Model Deployment at Scale on Kubernetes 🦄️

    TypeScript 818 73

  4. BentoVLLM BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    Python 138 17

  5. BentoDiffusion BentoDiffusion Public

    BentoDiffusion: A collection of diffusion models served with BentoML

    Python 373 28

  6. comfy-pack comfy-pack Public

    A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

    Python 170 23

Repositories

Showing 10 of 107 repositories
  • OpenLLM Public

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 11,618 Apache-2.0 754 3 3 Updated Jul 28, 2025
  • helm-charts Public
    bentoml/helm-charts’s past year of commit activity
    1 0 0 0 Updated Jul 28, 2025
  • yatai-image-builder Public

    🐳 Build OCI images for Bentos in k8s

    bentoml/yatai-image-builder’s past year of commit activity
    Go 19 12 6 13 Updated Jul 28, 2025
  • CLIP-API-service Public

    CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

    bentoml/CLIP-API-service’s past year of commit activity
    Jupyter Notebook 66 Apache-2.0 4 0 1 Updated Jul 28, 2025
  • llm-inference-handbook Public

    Everything you need to know about LLM inference

    bentoml/llm-inference-handbook’s past year of commit activity
    TypeScript 202 MIT 17 3 0 Updated Jul 27, 2025
  • BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    bentoml/BentoML’s past year of commit activity
    Python 7,929 Apache-2.0 859 131 4 Updated Jul 28, 2025
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 138 Apache-2.0 17 0 0 Updated Jul 24, 2025
  • comfy-pack Public

    A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

    bentoml/comfy-pack’s past year of commit activity
    Python 170 Apache-2.0 23 10 1 Updated Jul 23, 2025
  • bentoml/bentocloud-homepage-news’s past year of commit activity
    3 2 0 0 Updated Jul 8, 2025
  • BentoSpacy Public
    bentoml/BentoSpacy’s past year of commit activity
    Python 0 0 0 0 Updated Jul 7, 2025