LocalLab lets you run AI language models on your computer or Google Colab - no cloud services needed! Think of it as having ChatGPT-like capabilities right on your machine.
LocalLab consists of two parts working together:
graph LR
A[LocalLab Server] -->|Runs| B[AI Models]
C[Your Code] -->|Uses| D[LocalLab Client] -->|Talks to| A
Think of the server as your personal AI engine. It:
- Downloads and runs AI models on your computer
- Manages memory and resources automatically
- Optimizes performance based on your hardware
- Provides a simple API for accessing models
You can run it:
- On your computer (local mode)
- On Google Colab (free GPU mode)
The client is how your code talks to the AI. It:
- Connects to your LocalLab server
- Sends requests for text generation
- Handles chat conversations
- Processes multiple requests at once
- Streams responses in real-time
When you use LocalLab:
-
Server Setup
from locallab import start_server start_server() # Server starts and loads AI model
-
Client Connection
from locallab.client import LocalLabClient server_url = "http://localhost:8000" # or "https://your-ngrok-url.ngrok.app" client = LocalLabClient(server_url)
-
AI Interaction
# Your code sends requests through the client response = await client.generate("Write a story") print(response) # Server processes and returns AI response
# Generate text
response = await client.generate("Hello!")
# Chat with AI
response = await client.chat([
{"role": "user", "content": "Hi!"}
])
# Process multiple prompts
responses = await client.batch_generate([
"Write a joke",
"Tell a story"
])Local Computer:
- Python 3.8+
- 4GB RAM minimum
- GPU optional (but recommended)
Google Colab:
- Just a Google account!
- Free tier works fine
New to AI/Programming?
- Start with our Getting Started Guide
- Try the Basic Examples
- Join our Community
Developer?
- Check API Reference
- See Client Libraries
- Read Advanced Features
Our Documentation Guide will help you:
- Understand LocalLab's features
- Learn best practices
- Find solutions to common issues
- Master advanced features
- Easy Setup: Just pip install and run
- Multiple Models: Use any Hugging Face model
- Resource Efficient: Automatic optimization
- Privacy First: All local, no data sent to cloud
- Free GPU: Google Colab integration
- Check FAQ
- Visit Troubleshooting
- Ask in Discussions