FastAPI-BitNet #110
grctest
started this conversation in
Show and tell
FastAPI-BitNet
#110
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've created the project FastAPI-BitNet which uses Uvicorn + FastAPI (Python) and Docker to provide a reliable REST API for testing BitNet out locally!
It supports running the inference framework, running bitnet model benchmarks and calculating bitnet model perplexity values.
https://github.com/grctest/FastAPI-BitNet
Beta Was this translation helpful? Give feedback.
All reactions