LLM

Description

This repo allows you to create LLM Model API which runs on Google Colab.

With this LLM can be used in the following manner

$ llm = LLM()

• Currently, the Quantized Version of LLAMA2 is hardcoded within the code but can be customized to support any model.
• I'll add a tutorial on how to use vector embeddings as well maybe in the same repo or other repo.

How to use it?

1. Run Server

Don't forget to turn on the GPU.

2. Run notebook

Replace the URL in notebook with the one generated by server. And enjoy