fix: Plugin System Refactor

backends/advanced/Docs/plugin-development-guide.md

@@ -24,11 +24,6 @@ Chronicle's plugin system allows you to extend functionality by subscribing to e
 - **Configurable**: YAML-based configuration with environment variable support
 - **Isolated**: Each plugin runs independently with proper error handling
 
-### Plugin Types
-
-- **Core Plugins**: Built-in plugins (`homeassistant`, `test_event`)
-- **Community Plugins**: Auto-discovered plugins in `plugins/` directory
-
 ## Quick Start
 
 ### 1. Generate Plugin Boilerplate
@@ -207,6 +202,84 @@ async def on_memory_processed(self, context: PluginContext):
         await self.index_memory(memory)
 ```
 
+### 4. Button Events (`button.single_press`, `button.double_press`)
+
+**When**: OMI device button is pressed
+**Context Data**:
+- `state` (str): Button state (`SINGLE_TAP`, `DOUBLE_TAP`)
+- `timestamp` (float): Unix timestamp of the event
+- `audio_uuid` (str): Current audio session UUID (may be None)
+- `session_id` (str): Streaming session ID (for conversation close)
+- `client_id` (str): Client device identifier
+
+**Data Flow**:
+```
+OMI Device (BLE)
+  → Button press on physical device
+  → BLE characteristic notifies with 8-byte payload
+  ↓
+friend-lite-sdk (extras/friend-lite-sdk/)
+  → parse_button_event() converts payload → ButtonState IntEnum
+  ↓
+BLE Client (extras/local-omi-bt/ or mobile app)
+  → Formats as Wyoming protocol: {"type": "button-event", "data": {"state": "SINGLE_TAP"}}
+  → Sends over WebSocket
+  ↓
+Backend (websocket_controller.py)
+  → _handle_button_event() stores marker on client_state
+  → Maps ButtonState → PluginEvent using enums (plugins/events.py)
+  → Dispatches granular event to plugin system
+  ↓
+Plugin System
+  → Routed to subscribed plugins (e.g., test_button_actions)
+  → Plugins use PluginServices for system actions and cross-plugin calls
+```
+
+**Use Cases**:
+- Close current conversation (single press)
+- Toggle smart home devices (double press)
+- Custom actions via cross-plugin communication
+
+**Example**:
+```python
+async def on_button_event(self, context: PluginContext):
+    if context.event == PluginEvent.BUTTON_SINGLE_PRESS:
+        session_id = context.data.get('session_id')
+        await context.services.close_conversation(session_id)
+```
+
+### 5. Plugin Action Events (`plugin_action`)
+
+**When**: Another plugin calls `context.services.call_plugin()`
+**Context Data**:
+- `action` (str): Action name (e.g., `toggle_lights`)
+- Plus any additional data from the calling plugin
+
+**Use Cases**:
+- Cross-plugin communication (button press → toggle lights)
+- Service orchestration between plugins
+
+**Example**:
+```python
+async def on_plugin_action(self, context: PluginContext):
+    action = context.data.get('action')
+    if action == 'toggle_lights':
+        # Handle the action
+        ...
+```
+
+### PluginServices
+
+Plugins receive a `services` object on the context for system and cross-plugin interaction:
+
+```python
+# Close the current conversation (triggers post-processing)
+await context.services.close_conversation(session_id, reason)
+
+# Call another plugin's on_plugin_action() handler
+result = await context.services.call_plugin("homeassistant", "toggle_lights", data)
+```
+
 ## Creating Your First Plugin
 
 ### Step 1: Generate Boilerplate
@@ -225,7 +298,7 @@ import logging
 import re
 from typing import Any, Dict, List, Optional
 
-from ..base import BasePlugin, PluginContext, PluginResult
+from advanced_omi_backend.plugins.base import BasePlugin, PluginContext, PluginResult
 
 logger = logging.getLogger(__name__)
 
@@ -671,7 +744,7 @@ async def on_conversation_complete(self, context):
 **Solution**:
 - Restart backend after adding dependencies
 - Verify imports are from correct modules
-- Check relative imports use `..base` for base classes
+- Use absolute imports for framework classes: `from advanced_omi_backend.plugins.base import BasePlugin`
 
 ### Database Connection Issues
 
@@ -749,12 +822,13 @@ class ExternalServicePlugin(BasePlugin):
 
 ## Resources
 
-- **Base Plugin Class**: `backends/advanced/src/advanced_omi_backend/plugins/base.py`
-- **Example Plugins**:
+- **Plugin Framework**: `backends/advanced/src/advanced_omi_backend/plugins/` (base.py, router.py, events.py, services.py)
+- **Plugin Implementations**: `plugins/` at repo root
   - Email Summarizer: `plugins/email_summarizer/`
   - Home Assistant: `plugins/homeassistant/`
   - Test Event: `plugins/test_event/`
-- **Plugin Generator**: `scripts/create_plugin.py`
+  - Test Button Actions: `plugins/test_button_actions/`
+- **Plugin Generator**: `backends/advanced/scripts/create_plugin.py`
 - **Configuration**: `config/plugins.yml.template`
 
 ## Contributing Plugins

backends/advanced/docker-compose-test.yml

@@ -21,6 +21,7 @@ services:
       - ../../config:/app/config  # Mount config directory with defaults.yml
       - ../../tests/configs:/app/test-configs:ro  # Mount test-specific configs
       - ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml  # Mount test plugins config to correct location
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       # Override with test-specific settings
       - MONGODB_URI=mongodb://mongo-test:27017/test_db
@@ -223,6 +224,7 @@ services:
       - ../../config:/app/config  # Mount config directory with defaults.yml
       - ../../tests/configs:/app/test-configs:ro  # Mount test-specific configs
       - ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml  # Mount test plugins config to correct location
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       # Same environment as backend
       - MONGODB_URI=mongodb://mongo-test:27017/test_db

backends/advanced/docker-compose.yml

@@ -40,6 +40,7 @@ services:
       - ./data/debug_dir:/app/debug_dir
       - ./data:/app/data
       - ../../config:/app/config  # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       - DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
       - PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
@@ -95,6 +96,7 @@ services:
       - ./data/audio_chunks:/app/audio_chunks
       - ./data:/app/data
       - ../../config:/app/config  # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       - DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
       - PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
@@ -212,8 +214,8 @@ services:
       - "6033:6033" # gRPC
       - "6034:6034" # HTTP
     volumes:
-      - ./data/qdrant_data:/qdrant/storage 
-
+      - ./data/qdrant_data:/qdrant/storage
+    restart: unless-stopped
 
   mongo:
     image: mongo:8.0.14
@@ -227,6 +229,7 @@ services:
       timeout: 5s
       retries: 5
       start_period: 10s
+    restart: unless-stopped
 
   redis:
     image: redis:7-alpine
@@ -235,6 +238,7 @@ services:
     volumes:
       - ./data/redis_data:/data
     command: redis-server --appendonly yes
+    restart: unless-stopped
     healthcheck:
       test: ["CMD", "redis-cli", "ping"]
       interval: 5s

backends/advanced/scripts/create_plugin.py

@@ -37,10 +37,10 @@ def create_plugin(plugin_name: str, force: bool = False):
     # Convert to class name
     class_name = snake_to_pascal(plugin_name) + 'Plugin'
 
-    # Get plugins directory
+    # Get plugins directory (repo root plugins/)
     script_dir = Path(__file__).parent
     backend_dir = script_dir.parent
-    plugins_dir = backend_dir / 'src' / 'advanced_omi_backend' / 'plugins'
+    plugins_dir = backend_dir.parent.parent / 'plugins'
     plugin_dir = plugins_dir / plugin_name
 
     # Check if plugin already exists
@@ -83,7 +83,7 @@ def create_plugin(plugin_name: str, force: bool = False):
 import logging
 from typing import Any, Dict, List, Optional
 
-from ..base import BasePlugin, PluginContext, PluginResult
+from advanced_omi_backend.plugins.base import BasePlugin, PluginContext, PluginResult
 
 logger = logging.getLogger(__name__)
 

backends/advanced/src/advanced_omi_backend/client.py

@@ -51,6 +51,9 @@ def __init__(
         # NOTE: Removed in-memory transcript storage for single source of truth
         # Transcripts are stored only in MongoDB via TranscriptionManager
 
+        # Markers (e.g., button events) collected during the session
+        self.markers: List[dict] = []
+
         # Track if conversation has been closed
         self.conversation_closed: bool = False
 
@@ -102,6 +105,10 @@ def update_transcript_received(self):
         """Update timestamp when transcript is received (for timeout detection)."""
         self.last_transcript_time = time.time()
 
+    def add_marker(self, marker: dict) -> None:
+        """Add a marker (e.g., button event) to the current session."""
+        self.markers.append(marker)
+
     def should_start_new_conversation(self) -> bool:
         """Check if we should start a new conversation based on timeout."""
         if self.last_transcript_time is None:
@@ -114,8 +121,7 @@ def should_start_new_conversation(self) -> bool:
         return time_since_last_transcript > timeout_seconds
 
     async def close_current_conversation(self):
-        """Close the current conversation and queue necessary processing."""
-        # Prevent double closure
+        """Clean up in-memory speech segments for the current conversation."""
         if self.conversation_closed:
             audio_logger.debug(
                 f"🔒 Conversation already closed for client {self.client_id}, skipping"
@@ -125,23 +131,15 @@ async def close_current_conversation(self):
         self.conversation_closed = True
 
         if not self.current_audio_uuid:
-            audio_logger.info(f"🔒 No active conversation to close for client {self.client_id}")
             return
 
-        # NOTE: ClientState is legacy V1 code. In V2 architecture, conversation closure
-        # is handled by the websocket controllers using RQ jobs directly.
-        # This method is kept minimal for backward compatibility.
+        audio_logger.info(f"🔒 Closing conversation state for client {self.client_id}")
 
-        audio_logger.info(f"🔒 Closing conversation for client {self.client_id}, audio_uuid: {self.current_audio_uuid}")
-
-        # Clean up speech segments for this conversation
         if self.current_audio_uuid in self.speech_segments:
             del self.speech_segments[self.current_audio_uuid]
         if self.current_audio_uuid in self.current_speech_start:
             del self.current_speech_start[self.current_audio_uuid]
 
-        audio_logger.info(f"✅ Cleaned up state for {self.current_audio_uuid}")
-
     async def start_new_conversation(self):
         """Start a new conversation by closing current and resetting state."""
         await self.close_current_conversation()
@@ -151,11 +149,9 @@ async def start_new_conversation(self):
         self.conversation_start_time = time.time()
         self.last_transcript_time = None
         self.conversation_closed = False
+        self.markers = []
 
-        audio_logger.info(
-            f"Client {self.client_id}: Started new conversation due to "
-            f"{NEW_CONVERSATION_TIMEOUT_MINUTES}min timeout"
-        )
+        audio_logger.info(f"Client {self.client_id}: Started new conversation")
 
     async def disconnect(self):
         """Clean disconnect of client state."""