Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
5ea13cc
refactor memory
0xrushi Dec 18, 2025
9a372f5
Merge branch 'main' into fix/cleanup-model
0xrushi Dec 18, 2025
649c0de
add config
0xrushi Dec 18, 2025
4597b80
merge
0xrushi Dec 18, 2025
ab89263
Merge branch 'fix/cleanup-model' of github.com:0xrushi/friend-lite in…
0xrushi Dec 18, 2025
6976b6f
docstring
0xrushi Dec 18, 2025
77fea64
more cleanup
0xrushi Dec 18, 2025
c9b0590
code quality
0xrushi Dec 18, 2025
9816c2f
code quality
0xrushi Dec 18, 2025
0464b32
unused return
0xrushi Dec 18, 2025
5f325c1
DOTTED GET
0xrushi Dec 18, 2025
686f357
Merge branch 'dev' into fix/cleanup-model
AnkushMalaker Dec 18, 2025
43ed63e
Refactor Docker and CI configurations
AnkushMalaker Dec 19, 2025
db83fe4
Merge branch 'dev' into fix/cleanup-model
AnkushMalaker Dec 20, 2025
790ba69
Update configuration files for model providers and Docker setup
AnkushMalaker Dec 20, 2025
3608383
Merge pull request #220 from chronicler-ai/fix/cleanup-model-2
AnkushMalaker Dec 20, 2025
b9642c8
Update transcription job handling to format speaker IDs
AnkushMalaker Dec 22, 2025
a8e8bcc
Remove loading of backend .env file in test environment setup
AnkushMalaker Dec 22, 2025
f3701c7
Enhance configuration management and setup wizard
AnkushMalaker Dec 22, 2025
5de2dc9
Enhance HTTPS configuration in setup wizard
AnkushMalaker Dec 22, 2025
7f77600
Add source parameter to audio file writing in websocket controller
AnkushMalaker Dec 22, 2025
b6ba9ea
Merge branch 'fix/cleanup-model' into feat/config-yaml-compatible-wizard
AnkushMalaker Dec 22, 2025
274a38f
Merge pull request #227 from chronicler-ai/feat/config-yaml-compatibl…
AnkushMalaker Dec 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/robot-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,6 @@ jobs:
echo "LLM_PROVIDER: $LLM_PROVIDER"
echo "TRANSCRIPTION_PROVIDER: $TRANSCRIPTION_PROVIDER"

# Create memory_config.yaml from template (file is gitignored)
echo "Creating memory_config.yaml from template..."
cp memory_config.yaml.template memory_config.yaml

# Clean any existing test containers for fresh start
echo "Cleaning up any existing test containers..."
docker compose -f docker-compose-test.yml down -v || true
Expand Down
12 changes: 6 additions & 6 deletions Docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ curl -X POST "http://localhost:8000/api/process-audio-files" \

**Implementation**:
- **Memory System**: `src/advanced_omi_backend/memory/memory_service.py` + `src/advanced_omi_backend/controllers/memory_controller.py`
- **Configuration**: `memory_config.yaml` + `src/advanced_omi_backend/memory_config_loader.py`
- **Configuration**: memory settings in `config.yml` (memory section)

### Authentication & Security
- **Email Authentication**: Login with email and password
Expand Down Expand Up @@ -541,10 +541,10 @@ OPENMEMORY_MCP_URL=http://host.docker.internal:8765

> 🎯 **New to memory configuration?** Read our [Memory Configuration Guide](./memory-configuration-guide.md) for a step-by-step setup guide with examples.

The system uses **centralized configuration** via `memory_config.yaml` for all memory extraction settings. All hardcoded values have been removed from the code to ensure consistent, configurable behavior.
The system uses **centralized configuration** via `config.yml` for all models (LLM, embeddings, vector store) and memory extraction settings.

### Configuration File Location
- **Path**: `backends/advanced-backend/memory_config.yaml`
- **Path**: repository `config.yml` (override with `CONFIG_FILE` env var)
- **Hot-reload**: Changes are applied on next processing cycle (no restart required)
- **Fallback**: If file is missing, system uses safe defaults with environment variables

Expand Down Expand Up @@ -613,7 +613,7 @@ If you experience JSON parsing errors in fact extraction:

2. **Enable fact extraction** with reliable JSON output:
```yaml
# In memory_config.yaml
# In config.yml (memory section)
fact_extraction:
enabled: true # Safe to enable with GPT-4o
```
Expand Down Expand Up @@ -727,5 +727,5 @@ curl -H "Authorization: Bearer $ADMIN_TOKEN" \
- **Connect audio clients** using the WebSocket API
- **Explore the dashboard** to manage conversations and users
- **Review the user data architecture** for understanding data organization
- **Customize memory extraction** by editing `memory_config.yaml`
- **Monitor processing performance** using debug API endpoints
- **Customize memory extraction** by editing the `memory` section in `config.yml`
- **Monitor processing performance** using debug API endpoints
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Self-hostable AI system that captures audio/video data from OMI devices and othe

## Quick Start → [Get Started](quickstart.md)

Clone, run setup wizard, start services, access at http://localhost:5173
Run setup wizard, start services, access at http://localhost:5173

## Screenshots

Expand Down
2 changes: 1 addition & 1 deletion backends/advanced/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ SPEECH_INACTIVITY_THRESHOLD_SECONDS=60 # Close conversation after N seconds of
# When enabled, only creates conversations when enrolled speakers are detected
# Requires speaker recognition service to be running and speakers to be enrolled
# Set to "true" to enable, "false" or omit to disable
RECORD_ONLY_ENROLLED_SPEAKERS=true
RECORD_ONLY_ENROLLED_SPEAKERS=false

# ========================================
# DATABASE CONFIGURATION
Expand Down
2 changes: 1 addition & 1 deletion backends/advanced/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ COPY . .

# Copy configuration files if they exist, otherwise they will be created from templates at runtime
# The files are expected to exist, but we handle the case where they don't gracefully
COPY memory_config.yaml* ./

COPY diarization_config.json* ./


Expand Down
2 changes: 1 addition & 1 deletion backends/advanced/Dockerfile.k8s
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ RUN uv sync --extra deepgram
COPY . .

# Copy memory config (created by init.sh from template)
COPY memory_config.yaml ./


# Copy and make K8s startup scripts executable
COPY start-k8s.sh start-workers.sh ./
Expand Down
28 changes: 14 additions & 14 deletions backends/advanced/Docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Welcome to chronicle! This guide provides the optimal reading sequence to unders
- What the system does (voice → memories)
- Key features and capabilities
- Basic setup and configuration
- **Code References**: `src/advanced_omi_backend/main.py`, `memory_config.yaml`, `docker-compose.yml`
- **Code References**: `src/advanced_omi_backend/main.py`, `config.yml`, `docker-compose.yml`

### 2. **[System Architecture](./architecture.md)**
**Read second** - Complete technical architecture with diagrams
Expand Down Expand Up @@ -70,12 +70,12 @@ Welcome to chronicle! This guide provides the optimal reading sequence to unders

## 🔍 **Configuration & Customization**

### 6. **Configuration File** → `../memory_config.yaml`
### 6. **Configuration File** → `../config.yml`
**Central configuration for all extraction**
- Memory extraction settings and prompts
- Quality control and debug settings
- **Code References**:
- `src/advanced_omi_backend/memory_config_loader.py` (config loading)
- `src/advanced_omi_backend/model_registry.py` (config loading)
- `src/advanced_omi_backend/memory/memory_service.py` (config usage)
Comment on lines +73 to 79
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use proper heading syntax instead of emphasis.

Line 73 uses emphasis (bold) instead of a proper Markdown heading (line 72 is a ### heading, but line 73 should not be emphasized text pointing to a heading). Apply proper heading hierarchy.

Static analysis concern: markdownlint MD036 flags this pattern. Consider:

### Configuration File

Central configuration in `../config.yml` at repository root.
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)

74-74: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

🤖 Prompt for AI Agents
In backends/advanced/Docs/README.md around lines 73 to 79, the heading uses
emphasis markup inside the heading (e.g. "### 6. **Configuration File** →
`../config.yml`"); replace the emphasized text with a proper markdown heading
and clean heading hierarchy (e.g. "### Configuration File — ../config.yml" or
"### Configuration File" with the file path inline or on the next line), remove
the bold formatting, and ensure the following descriptive line is plain
paragraph text; this will satisfy markdownlint MD036 and keep the heading level
consistent.


---
Expand All @@ -86,11 +86,11 @@ Welcome to chronicle! This guide provides the optimal reading sequence to unders
1. [quickstart.md](./quickstart.md) - System overview
2. [architecture.md](./architecture.md) - Technical architecture
3. `src/advanced_omi_backend/main.py` - Core imports and setup
4. `memory_config.yaml` - Configuration overview
4. `config.yml` - Configuration overview

### **"I want to work on memory extraction"**
1. [memories.md](./memories.md) - Memory system details
2. `../memory_config.yaml` - Memory configuration
2. `../config.yml` - Models and memory configuration
3. `src/advanced_omi_backend/memory/memory_service.py` - Implementation
4. `src/advanced_omi_backend/controllers/memory_controller.py` - Processing triggers

Expand Down Expand Up @@ -128,9 +128,9 @@ backends/advanced-backend/
│ ├── controllers/ # Business logic controllers
│ ├── memory/
│ │ └── memory_service.py # Memory system (Mem0)
│ └── memory_config_loader.py # Configuration loading
│ └── model_registry.py # Configuration loading
├── memory_config.yaml # 📋 Central configuration
├── config.yml # 📋 Central configuration
├── MEMORY_DEBUG_IMPLEMENTATION.md # Debug system details
```

Expand All @@ -147,8 +147,8 @@ backends/advanced-backend/
- **Memories**: `src/advanced_omi_backend/memory/memory_service.py` → Mem0 → Qdrant

### **Configuration**
- **Loading**: `src/advanced_omi_backend/memory_config_loader.py`
- **File**: `memory_config.yaml`
- **Loading**: `src/advanced_omi_backend/model_registry.py`
- **File**: `config.yml`
- **Usage**: `src/advanced_omi_backend/memory/memory_service.py`

### **Authentication**
Expand All @@ -162,7 +162,7 @@ backends/advanced-backend/

1. **Follow the references**: Each doc links to specific code files and line numbers
2. **Use the debug API**: `GET /api/debug/memory/stats` shows live system status
3. **Check configuration first**: Many behaviors are controlled by `memory_config.yaml`
3. **Check configuration first**: Many behaviors are controlled by `config.yml`
4. **Understand the memory pipeline**: Memories (end-of-conversation)
5. **Test with curl**: All API endpoints have curl examples in the docs

Expand All @@ -175,23 +175,23 @@ backends/advanced-backend/
1. **Set up the system**: Follow [quickstart.md](./quickstart.md) to get everything running
2. **Test the API**: Use the curl examples in the documentation to test endpoints
3. **Explore the debug system**: Check `GET /api/debug/memory/stats` to see live data
4. **Modify configuration**: Edit `memory_config.yaml` to see how it affects extraction
4. **Modify configuration**: Edit `config.yml` (memory section) to see how it affects extraction
5. **Read the code**: Start with `src/advanced_omi_backend/main.py` and follow the references in each doc

### **Contributing Guidelines**

- **Add code references**: When updating docs, include file paths and line numbers
- **Test your changes**: Use the debug API to verify your modifications work
- **Update configuration**: Add new settings to `memory_config.yaml` when needed
- **Update configuration**: Add new settings to `config.yml` when needed
- **Follow the architecture**: Keep memories in their respective services

### **Getting Help**

- **Debug API**: `GET /api/debug/memory/*` endpoints show real-time system status
- **Configuration**: Check `memory_config.yaml` for behavior controls
- **Configuration**: Check `config.yml` for behavior controls
- **Logs**: Check Docker logs with `docker compose logs chronicle-backend`
- **Documentation**: Each doc file links to relevant code sections

---

This documentation structure ensures you understand both the **big picture** and **implementation details** in a logical progression!
This documentation structure ensures you understand both the **big picture** and **implementation details** in a logical progression!
6 changes: 3 additions & 3 deletions backends/advanced/Docs/contribution.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
1. Docs/quickstart.md (15 min)
2. Docs/architecture.md (20 min)
3. main.py - just the imports and WebSocket sections (15 min)
4. memory_config.yaml (10 min)
4. config.yml (memory section) (10 min)

🔧 "I want to work on memory extraction"

1. Docs/quickstart.md → Docs/memories.md
2. memory_config.yaml (memory_extraction section)
2. config.yml (memory.extraction section)
3. main.py lines 1047-1065 (trigger)
4. main.py lines 1163-1195 (processing)
5. src/memory/memory_service.py
Expand All @@ -32,4 +32,4 @@
Data Flow

Audio → Transcription → Dual Processing
└─ Memory Pipeline (end-of-conversation)
└─ Memory Pipeline (end-of-conversation)
6 changes: 3 additions & 3 deletions backends/advanced/Docs/memories.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This document explains how to configure and customize the memory service in the
- **Repository Layer**: `src/advanced_omi_backend/conversation_repository.py` (clean data access)
- **Processing Manager**: `src/advanced_omi_backend/processors.py` (MemoryProcessor class)
- **Conversation Management**: `src/advanced_omi_backend/conversation_manager.py` (lifecycle coordination)
- **Configuration**: `memory_config.yaml` + `src/memory_config_loader.py`
- **Configuration**: `config.yml` (memory section) + `src/model_registry.py`

## Overview

Expand Down Expand Up @@ -180,7 +180,7 @@ OPENAI_MODEL=gpt-5-mini # Recommended for reliable JSON output
# OPENAI_MODEL=gpt-3.5-turbo # Budget option
```

Or configure via `memory_config.yaml`:
Or configure via `config.yml` (memory block):

```yaml
memory_extraction:
Expand Down Expand Up @@ -674,4 +674,4 @@ The new architecture ensures proper user isolation and simplifies admin debuggin

Both load all user memories and view all memories are helpful
Both views complement each other - the debug view helps you understand how the system is working, while the clean view
helps you understand what content is being stored.
helps you understand what content is being stored.
8 changes: 4 additions & 4 deletions backends/advanced/Docs/memory-configuration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ This guide helps you set up and configure the memory system for the Friend Advan

1. **Copy the template configuration**:
```bash
cp memory_config.yaml.template memory_config.yaml
Edit the `memory` section of `config.yml`.
```

2. **Edit memory_config.yaml** with your preferred settings:
2. **Edit `config.yml`** with your preferred settings in the `memory` section:
```yaml
memory:
provider: "mem0" # or "basic" for simpler setup
Expand Down Expand Up @@ -127,6 +127,6 @@ memory:

## Next Steps

- Configure action items detection in `memory_config.yaml`
- Configure action items detection in `config.yml` (memory.extraction)
- Set up custom prompt templates for your use case
- Monitor memory processing in the debug dashboard
- Monitor memory processing in the debug dashboard
12 changes: 6 additions & 6 deletions backends/advanced/Docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ curl -X POST "http://localhost:8000/api/audio/upload" \

**Implementation**:
- **Memory System**: `src/advanced_omi_backend/memory/memory_service.py` + `src/advanced_omi_backend/controllers/memory_controller.py`
- **Configuration**: `memory_config.yaml` + `src/advanced_omi_backend/memory_config_loader.py`
- **Configuration**: `config.yml` (memory + models) in repo root

### Authentication & Security
- **Email Authentication**: Login with email and password
Expand Down Expand Up @@ -539,10 +539,10 @@ OPENMEMORY_MCP_URL=http://host.docker.internal:8765

> 🎯 **New to memory configuration?** Read our [Memory Configuration Guide](./memory-configuration-guide.md) for a step-by-step setup guide with examples.

The system uses **centralized configuration** via `memory_config.yaml` for all memory extraction settings. All hardcoded values have been removed from the code to ensure consistent, configurable behavior.
The system uses **centralized configuration** via `config.yml` for all memory extraction and model settings.

### Configuration File Location
- **Path**: `backends/advanced-backend/memory_config.yaml`
- **Path**: `config.yml` in repo root
- **Hot-reload**: Changes are applied on next processing cycle (no restart required)
- **Fallback**: If file is missing, system uses safe defaults with environment variables

Expand Down Expand Up @@ -611,7 +611,7 @@ If you experience JSON parsing errors in fact extraction:

2. **Enable fact extraction** with reliable JSON output:
```yaml
# In memory_config.yaml
# In config.yml (memory section)
fact_extraction:
enabled: true # Safe to enable with GPT-4o
```
Expand Down Expand Up @@ -725,5 +725,5 @@ curl -H "Authorization: Bearer $ADMIN_TOKEN" \
- **Connect audio clients** using the WebSocket API
- **Explore the dashboard** to manage conversations and users
- **Review the user data architecture** for understanding data organization
- **Customize memory extraction** by editing `memory_config.yaml`
- **Monitor processing performance** using debug API endpoints
- **Customize memory extraction** by editing the `memory` section in `config.yml`
- **Monitor processing performance** using debug API endpoints
4 changes: 2 additions & 2 deletions backends/advanced/SETUP_SCRIPTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This document explains the different setup scripts available in Friend-Lite and

| Script | Purpose | When to Use |
|--------|---------|-------------|
| `init.py` | **Main interactive setup wizard** | **Recommended for all users** - First time setup with guided configuration (located at repo root) |
| `init.py` | **Main interactive setup wizard** | **Recommended for all users** - First time setup with guided configuration (located at repo root). Memory now configured in `config.yml`. |
| `setup-https.sh` | HTTPS certificate generation | **Optional** - When you need secure connections for microphone access |

## Main Setup Script: `init.py`
Expand Down Expand Up @@ -157,4 +157,4 @@ Setup scripts are located as follows:
✅ **Provider selection**: Choose best services for your needs
✅ **Complete configuration**: Creates working .env with all settings
✅ **Next steps guidance**: Clear instructions for starting services
✅ **No manual editing**: Reduces errors from manual .env editing
✅ **No manual editing**: Reduces errors from manual .env editing
10 changes: 4 additions & 6 deletions backends/advanced/docker-compose-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ services:
- ./data/test_audio_chunks:/app/audio_chunks
- ./data/test_debug_dir:/app/debug_dir
- ./data/test_data:/app/data
- ../../config.yml:/app/config.yml:ro # Mount config.yml for model registry and memory settings
environment:
# Override with test-specific settings
- MONGODB_URI=mongodb://mongo-test:27017/test_db
Expand All @@ -24,10 +25,7 @@ services:
# Import API keys from environment
- DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- OPENAI_BASE_URL=https://api.openai.com/v1
# LLM provider configuration (required for memory service)
- LLM_PROVIDER=${LLM_PROVIDER:-openai}
- OPENAI_MODEL=${OPENAI_MODEL:-gpt-4o-mini}
- GROQ_API_KEY=${GROQ_API_KEY}
# Authentication (test-specific)
- AUTH_SECRET_KEY=test-jwt-signing-key-for-integration-tests
- ADMIN_PASSWORD=test-admin-password-123
Expand Down Expand Up @@ -131,6 +129,7 @@ services:
- ./data/test_audio_chunks:/app/audio_chunks
- ./data/test_debug_dir:/app/debug_dir
- ./data/test_data:/app/data
- ../../config.yml:/app/config.yml:ro # Mount config.yml for model registry and memory settings
environment:
# Same environment as backend
- MONGODB_URI=mongodb://mongo-test:27017/test_db
Expand All @@ -140,8 +139,7 @@ services:
- DEBUG_DIR=/app/debug_dir
- DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- LLM_PROVIDER=${LLM_PROVIDER:-openai}
- OPENAI_MODEL=${OPENAI_MODEL:-gpt-4o-mini}
- GROQ_API_KEY=${GROQ_API_KEY}
- AUTH_SECRET_KEY=test-jwt-signing-key-for-integration-tests
- ADMIN_PASSWORD=test-admin-password-123
- ADMIN_EMAIL=test-admin@example.com
Expand Down
Loading
Loading