Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion backends/advanced/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,11 @@ NEO4J_HOST=neo4j
NEO4J_USER=neo4j
NEO4J_PASSWORD=

# Langfuse API keys (for LLM observability)
# Langfuse (for LLM observability and prompt management)
LANGFUSE_HOST=
LANGFUSE_PUBLIC_KEY=
LANGFUSE_SECRET_KEY=
LANGFUSE_BASE_URL=http://langfuse-web:3000

# Tailscale auth key (for remote service access)
TS_AUTHKEY=
Expand Down
94 changes: 84 additions & 10 deletions backends/advanced/Docs/plugin-development-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,6 @@ Chronicle's plugin system allows you to extend functionality by subscribing to e
- **Configurable**: YAML-based configuration with environment variable support
- **Isolated**: Each plugin runs independently with proper error handling

### Plugin Types

- **Core Plugins**: Built-in plugins (`homeassistant`, `test_event`)
- **Community Plugins**: Auto-discovered plugins in `plugins/` directory

## Quick Start

### 1. Generate Plugin Boilerplate
Expand Down Expand Up @@ -207,6 +202,84 @@ async def on_memory_processed(self, context: PluginContext):
await self.index_memory(memory)
```

### 4. Button Events (`button.single_press`, `button.double_press`)

**When**: OMI device button is pressed
**Context Data**:
- `state` (str): Button state (`SINGLE_TAP`, `DOUBLE_TAP`)
- `timestamp` (float): Unix timestamp of the event
- `audio_uuid` (str): Current audio session UUID (may be None)
- `session_id` (str): Streaming session ID (for conversation close)
- `client_id` (str): Client device identifier

**Data Flow**:
```
OMI Device (BLE)
→ Button press on physical device
→ BLE characteristic notifies with 8-byte payload
friend-lite-sdk (extras/friend-lite-sdk/)
→ parse_button_event() converts payload → ButtonState IntEnum
BLE Client (extras/local-wearable-client/ or mobile app)
→ Formats as Wyoming protocol: {"type": "button-event", "data": {"state": "SINGLE_TAP"}}
→ Sends over WebSocket
Backend (websocket_controller.py)
→ _handle_button_event() stores marker on client_state
→ Maps ButtonState → PluginEvent using enums (plugins/events.py)
→ Dispatches granular event to plugin system
Plugin System
→ Routed to subscribed plugins (e.g., test_button_actions)
→ Plugins use PluginServices for system actions and cross-plugin calls
```

**Use Cases**:
- Close current conversation (single press)
- Toggle smart home devices (double press)
- Custom actions via cross-plugin communication

**Example**:
```python
async def on_button_event(self, context: PluginContext):
if context.event == PluginEvent.BUTTON_SINGLE_PRESS:
session_id = context.data.get('session_id')
await context.services.close_conversation(session_id)
```

### 5. Plugin Action Events (`plugin_action`)

**When**: Another plugin calls `context.services.call_plugin()`
**Context Data**:
- `action` (str): Action name (e.g., `toggle_lights`)
- Plus any additional data from the calling plugin

**Use Cases**:
- Cross-plugin communication (button press → toggle lights)
- Service orchestration between plugins

**Example**:
```python
async def on_plugin_action(self, context: PluginContext):
action = context.data.get('action')
if action == 'toggle_lights':
# Handle the action
...
```

### PluginServices

Plugins receive a `services` object on the context for system and cross-plugin interaction:

```python
# Close the current conversation (triggers post-processing)
await context.services.close_conversation(session_id, reason)

# Call another plugin's on_plugin_action() handler
result = await context.services.call_plugin("homeassistant", "toggle_lights", data)
```

## Creating Your First Plugin

### Step 1: Generate Boilerplate
Expand All @@ -225,7 +298,7 @@ import logging
import re
from typing import Any, Dict, List, Optional

from ..base import BasePlugin, PluginContext, PluginResult
from advanced_omi_backend.plugins.base import BasePlugin, PluginContext, PluginResult

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -671,7 +744,7 @@ async def on_conversation_complete(self, context):
**Solution**:
- Restart backend after adding dependencies
- Verify imports are from correct modules
- Check relative imports use `..base` for base classes
- Use absolute imports for framework classes: `from advanced_omi_backend.plugins.base import BasePlugin`

### Database Connection Issues

Expand Down Expand Up @@ -749,12 +822,13 @@ class ExternalServicePlugin(BasePlugin):

## Resources

- **Base Plugin Class**: `backends/advanced/src/advanced_omi_backend/plugins/base.py`
- **Example Plugins**:
- **Plugin Framework**: `backends/advanced/src/advanced_omi_backend/plugins/` (base.py, router.py, events.py, services.py)
- **Plugin Implementations**: `plugins/` at repo root
- Email Summarizer: `plugins/email_summarizer/`
- Home Assistant: `plugins/homeassistant/`
- Test Event: `plugins/test_event/`
- **Plugin Generator**: `scripts/create_plugin.py`
- Test Button Actions: `plugins/test_button_actions/`
- **Plugin Generator**: `backends/advanced/scripts/create_plugin.py`
- **Configuration**: `config/plugins.yml.template`

## Contributing Plugins
Expand Down
2 changes: 2 additions & 0 deletions backends/advanced/docker-compose-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ services:
- ../../config:/app/config # Mount config directory with defaults.yml
- ../../tests/configs:/app/test-configs:ro # Mount test-specific configs
- ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml # Mount test plugins config to correct location
- ../../plugins:/app/plugins # External plugins directory
environment:
# Override with test-specific settings
- MONGODB_URI=mongodb://mongo-test:27017/test_db
Expand Down Expand Up @@ -223,6 +224,7 @@ services:
- ../../config:/app/config # Mount config directory with defaults.yml
- ../../tests/configs:/app/test-configs:ro # Mount test-specific configs
- ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml # Mount test plugins config to correct location
- ../../plugins:/app/plugins # External plugins directory
environment:
# Same environment as backend
- MONGODB_URI=mongodb://mongo-test:27017/test_db
Expand Down
11 changes: 6 additions & 5 deletions backends/advanced/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ services:
- ./data/debug_dir:/app/debug_dir
- ./data:/app/data
- ../../config:/app/config # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
- ../../plugins:/app/plugins # External plugins directory
environment:
- DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
- PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
Expand Down Expand Up @@ -95,6 +96,7 @@ services:
- ./data/audio_chunks:/app/audio_chunks
- ./data:/app/data
- ../../config:/app/config # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
- ../../plugins:/app/plugins # External plugins directory
environment:
- DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
- PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
Expand Down Expand Up @@ -212,8 +214,8 @@ services:
- "6033:6033" # gRPC
- "6034:6034" # HTTP
volumes:
- ./data/qdrant_data:/qdrant/storage

- ./data/qdrant_data:/qdrant/storage
restart: unless-stopped

mongo:
image: mongo:8.0.14
Expand All @@ -227,6 +229,7 @@ services:
timeout: 5s
retries: 5
start_period: 10s
restart: unless-stopped

redis:
image: redis:7-alpine
Expand All @@ -235,6 +238,7 @@ services:
volumes:
- ./data/redis_data:/data
command: redis-server --appendonly yes
restart: unless-stopped
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
Expand Down Expand Up @@ -267,9 +271,6 @@ services:
timeout: 10s
retries: 5
start_period: 30s
profiles:
- obsidian
- knowledge-graph

# ollama:
# image: ollama/ollama:latest
Expand Down
Loading
Loading