Skip to content

An easy-to-use, high-performance(?) backend for serving LLMs and other AI models, built on FastAPI.

License

Notifications You must be signed in to change notification settings

fairyshine/FastMindAPI

Folders and files

NameName
Last commit message
Last commit date
Sep 19, 2024
Sep 14, 2024
Sep 25, 2024
Sep 28, 2024
Sep 27, 2024
Sep 27, 2024
Sep 14, 2024
Sep 27, 2024
Sep 27, 2024
Sep 24, 2024
Sep 23, 2024
Sep 14, 2024

Repository files navigation

FastMindAPI

PyPI - Version GitHub License GitHub code size in bytes PyPI - Downloads

An easy-to-use, high-performance(?) backend for serving LLMs and other AI models, built on FastAPI.

πŸš€ Quick Start

Install

pip install fastmindapi

Use

Run the server

# in Shell
fastmindapi-server --port 8000
# in Python
import fastmindapi as FM

server = FM.Server()
server.run()

Access via client / HTTP requests

curl http://IP:PORT/docs#/
import fastmindapi as FM

client = FM.Client(IP="x.x.x.x", PORT=xxx) # 127.0.0.1:8000 for default

client.add_model_info_list(model_info_list)
client.load_model(model_name)
client.generate(model_name, generation_request)

πŸͺ§ We primarily maintain the backend server; the client is provided for reference only. The main usage is through sending HTTP requests. (We might release FM-GUI in the future.)

✨ Features

Model: Support models with various backends

Modules: More than just chatting with models

  • Function Calling (extra tools in Python)
  • Retrieval
  • Agent
  • ...

Flexibility: Easy to Use & Highly Customizable

  • Load the model when coding / runtime
  • Add any APIs you want

About

An easy-to-use, high-performance(?) backend for serving LLMs and other AI models, built on FastAPI.

Resources

License

Stars

Watchers

Forks

Packages

No packages published