pre-release v0.2 (#293)

* Enhance ASR service descriptions and provider feedback in wizard.py

- Updated the description for the 'asr-services' to remove the specific mention of 'Parakeet', making it more general.
- Improved the console output for auto-selected services to include the transcription provider label, enhancing user feedback during service selection.

* Implement LangFuse integration for observability and prompt management

- Added LangFuse configuration options in the .env.template for observability and prompt management.
- Introduced setup_langfuse method in ChronicleSetup to handle LangFuse initialization and configuration prompts.
- Enhanced prompt management by integrating a centralized PromptRegistry for dynamic prompt retrieval and registration.
- Updated various services to utilize prompts from the PromptRegistry, improving flexibility and maintainability.
- Refactored OpenAI client initialization to support optional LangFuse tracing, enhancing observability during API interactions.
- Added new prompt defaults for memory management and conversation handling, ensuring consistent behavior across the application.

* Enhance LangFuse integration and service management

- Added LangFuse service configuration in services.py and wizard.py, including paths, commands, and descriptions.
- Implemented auto-selection for LangFuse during service setup, improving user experience.
- Enhanced service startup process to display prompt management tips for LangFuse, guiding users on editing AI prompts.
- Updated run_service_setup to handle LangFuse-specific parameters, including admin credentials and API keys, ensuring seamless integration with backend services.

* Feat/better reprocess memory (#300)

* Enhance ASR service descriptions and provider feedback in wizard.py (#290)

- Updated the description for the 'asr-services' to remove the specific mention of 'Parakeet', making it more general.
- Improved the console output for auto-selected services to include the transcription provider label, enhancing user feedback during service selection.

* Refactor Obsidian and Knowledge Graph integration in services and setup

- Removed redundant Obsidian and Knowledge Graph configuration checks from services.py, streamlining the command execution process.
- Updated wizard.py to enhance user experience by setting default options for speaker recognition during service selection.
- Improved Neo4j password handling in setup processes, ensuring consistent configuration prompts and feedback.
- Introduced a new cron scheduler for managing scheduled tasks, enhancing the backend's automation capabilities.
- Added new entity annotation features, allowing for corrections and updates to knowledge graph entities directly through the API.

* Enhance ASR services configuration and VibeVoice integration

- Added new configuration options for VibeVoice ASR in defaults.yml, including batching parameters for audio processing.
- Updated Docker Compose files to mount the config directory, ensuring access to ASR service configurations.
- Enhanced the VibeVoice transcriber to load configuration settings from defaults.yml, allowing for dynamic adjustments via environment variables.
- Introduced quantization options for model loading in the VibeVoice transcriber, improving performance and flexibility.
- Refactored the speaker identification process to streamline audio handling and improve logging for better debugging.
- Updated documentation to reflect new configuration capabilities and usage instructions for the VibeVoice ASR provider.

* Enhance LangFuse integration and memory reprocessing capabilities

- Introduced functions for checking LangFuse configuration in services.py, ensuring proper setup for observability.
- Updated wizard.py to facilitate user input for LangFuse configuration, including options for local and external setups.
- Implemented memory reprocessing logic in memory services to update existing memories based on speaker re-identification.
- Enhanced speaker recognition client to support per-segment identification, improving accuracy during reprocessing.
- Refactored various components to streamline handling of LangFuse parameters and improve overall service management.

* Enhance service management and user input handling

- Updated services.py to include LangFuse configuration checks during service startup, improving observability setup.
- Refactored wizard.py to utilize a masked input for Neo4j password prompts, enhancing user experience and security.
- Improved cron scheduler in advanced_omi_backend to manage active tasks and validate cron expressions, ensuring robust job execution.
- Enhanced speaker recognition client documentation to clarify user_id limitations, preparing for future multi-user support.
- Updated knowledge graph routes to enforce validation on entity updates, ensuring at least one field is provided for updates.

* fix: Plugin System Refactor (#301)

* Refactor connect-omi.py for improved device selection and user interaction

- Replaced references to the chronicle Bluetooth library with friend_lite for device management.
- Removed the list_devices function and implemented a new prompt_user_to_pick_device function to enhance user interaction when selecting OMI/Neo devices.
- Updated the find_and_set_omi_mac function to utilize the new device selection method, improving the overall flow of device connection.
- Added a new scan_devices.py script for quick scanning of neo/neosapien devices, enhancing usability.
- Updated README.md to reflect new usage instructions and prerequisites for connecting to OMI devices over Bluetooth.
- Enhanced start.sh to ensure proper environment variable setup for macOS users.

* Add friend-lite-sdk: Initial implementation of Python SDK for OMI/Friend Lite BLE devices

- Introduced the friend-lite-sdk, a Python SDK for OMI/Friend Lite BLE devices, enabling audio streaming, button events, and transcription functionalities.
- Added LICENSE and NOTICE files to clarify licensing and attribution.
- Created pyproject.toml for package management, specifying dependencies and project metadata.
- Developed core modules including bluetooth connection handling, button event parsing, audio decoding, and transcription capabilities.
- Implemented example usage in README.md to guide users on installation and basic functionality.
- Enhanced connect-omi.py to utilize the new SDK for improved device management and event handling.
- Updated requirements.txt to reference the new SDK for local development.

This commit lays the foundation for further enhancements and integrations with OMI devices.

* Enhance client state and plugin architecture for button event handling

- Introduced a new `markers` list in `ClientState` to collect button event data during sessions.
- Added `add_marker` method to facilitate the addition of markers to the current session.
- Implemented `on_button_event` method in the `BasePlugin` class to handle device button events, providing context data for button state and timestamps.
- Updated `PluginRouter` to route button events to the appropriate plugin handler.
- Enhanced conversation job handling to attach markers from Redis sessions, improving the tracking of button events during conversations.

* Move plugins locatino

- Introduced the Email Summarizer plugin that automatically sends email summaries upon conversation completion.
- Implemented SMTP email service for sending formatted HTML and plain text emails.
- Added configuration options for SMTP settings and email content in `config.yml`.
- Created setup script for easy configuration of SMTP credentials and plugin orchestration.
- Enhanced documentation with usage instructions and troubleshooting tips for the plugin.
- Updated existing plugin architecture to support new event handling for email summaries.

* Enhance Docker Compose and Plugin Management

- Added external plugins directory to Docker Compose files for better plugin management.
- Updated environment variables for MongoDB and Redis services to ensure consistent behavior.
- Introduced new dependencies in `uv.lock` for improved functionality.
- Refactored audio processing to support various audio formats and enhance error handling.
- Implemented new plugin event types and services for better integration and communication between plugins.
- Enhanced conversation and session management to support new closing mechanisms and event logging.

* Update audio processing and event logging

- Increased the maximum event log size in PluginRouter from 200 to 1000 for improved event tracking.
- Refactored audio stream producer to dynamically read audio format from Redis session metadata, enhancing flexibility in audio handling.
- Updated transcription job processing to utilize session-specific audio format settings, ensuring accurate audio processing.
- Enhanced audio file writing utility to accept PCM parameters, allowing for better control over audio data handling.

* Add markers list to ClientState and update timeout trigger comment

- Introduced a new `markers` list in `ClientState` to track button event data during conversations.
- Updated comment in `open_conversation_job` to clarify the behavior of the `timeout_triggered` variable, ensuring better understanding of session management.

* Refactor audio file logging and error handling

- Updated audio processing logs to consistently use the `filename` variable instead of `file.filename` for clarity.
- Enhanced error logging to utilize the `filename` variable, improving traceability of issues during audio processing.
- Adjusted title generation logic to handle cases where the filename is "unknown," ensuring a default title is used.
- Minor refactor in conversation closing logs to use `user.user_id` for better consistency in user identification.

* Enhance conversation retrieval with pagination and orphan handling

- Updated `get_conversations` function to support pagination through `limit` and `offset` parameters, improving performance for large datasets.
- Consolidated query logic to fetch both normal and orphan conversations in a single database call, reducing round-trips and enhancing efficiency.
- Modified the response structure to include total count, limit, and offset in the returned data for better client-side handling.
- Adjusted database indexing to optimize queries for paginated results, ensuring faster access to conversation data.

* Refactor connection logging in transcribe function

- Moved connection logging for the Wyoming server to a more structured format within the `transcribe_wyoming` function.
- Ensured that connection attempts and successes are logged consistently for better traceability during audio transcription processes.

* Feat/neo sdk (#302)

* Update friend-lite-sdk for Neo1 device support and enhance documentation

- Updated the friend-lite-sdk to version 0.3.0, reflecting the transition to support OMI/Neo1 BLE wearable devices.
- Refactored the Bluetooth connection handling to introduce a new `WearableConnection` class, enhancing the connection lifecycle management for wearable devices.
- Added a new `Neo1Connection` class for controlling Neo1 devices, including methods for sleep and wake functionalities.
- Updated UUID constants to include Neo1-specific characteristics, improving device interaction capabilities.
- Revised the plugin development guide to reflect changes in device naming and connection processes.
- Removed outdated local OMI Bluetooth scripts and documentation to streamline the project structure and focus on wearable client development.

* Refactor backend audio streaming to use Opus codec and enhance menu app functionality

- Updated backend_sender.py to stream raw Opus audio instead of PCM, improving bandwidth efficiency.
- Modified stream_to_backend function to handle Opus audio data and adjusted audio chunk parameters accordingly.
- Enhanced main.py with new CLI commands for device scanning and connection management, improving user experience.
- Introduced menu_app.py for a macOS menu bar application, providing a user-friendly interface for device management and status display.
- Added README.md to document usage instructions and configuration details for the local wearable client.
- Updated requirements.txt to include new dependencies for the menu app and service management.
- Implemented service.py for managing launchd service installation and configuration on macOS, enabling auto-start on login.

* Refactor audio processing and queue management in local wearable client

- Removed the audio queue in favor of a dedicated BLE data queue and backend queue for improved data handling.
- Enhanced the `connect_and_stream` function to streamline audio decoding and writing to the local file sink.
- Updated the handling of BLE data to ensure robust queue management and error logging.
- Improved task management during device disconnection to ensure proper cleanup and error handling.
- Updated requirements.txt to specify a minimum version for easy_audio_interfaces, ensuring compatibility.

Author: AnkushMalaker

backends/advanced/.env.template

@@ -53,9 +53,11 @@ NEO4J_HOST=neo4j
 NEO4J_USER=neo4j
 NEO4J_PASSWORD=
 
-# Langfuse API keys (for LLM observability)
+# Langfuse (for LLM observability and prompt management)
+LANGFUSE_HOST=
 LANGFUSE_PUBLIC_KEY=
 LANGFUSE_SECRET_KEY=
+LANGFUSE_BASE_URL=http://langfuse-web:3000
 
 # Tailscale auth key (for remote service access)
 TS_AUTHKEY=

backends/advanced/Docs/plugin-development-guide.md

@@ -24,11 +24,6 @@ Chronicle's plugin system allows you to extend functionality by subscribing to e
 - **Configurable**: YAML-based configuration with environment variable support
 - **Isolated**: Each plugin runs independently with proper error handling
 
-### Plugin Types
-
-- **Core Plugins**: Built-in plugins (`homeassistant`, `test_event`)
-- **Community Plugins**: Auto-discovered plugins in `plugins/` directory
-
 ## Quick Start
 
 ### 1. Generate Plugin Boilerplate
@@ -207,6 +202,84 @@ async def on_memory_processed(self, context: PluginContext):
         await self.index_memory(memory)
 ```
 
+### 4. Button Events (`button.single_press`, `button.double_press`)
+
+**When**: OMI device button is pressed
+**Context Data**:
+- `state` (str): Button state (`SINGLE_TAP`, `DOUBLE_TAP`)
+- `timestamp` (float): Unix timestamp of the event
+- `audio_uuid` (str): Current audio session UUID (may be None)
+- `session_id` (str): Streaming session ID (for conversation close)
+- `client_id` (str): Client device identifier
+
+**Data Flow**:
+```
+OMI Device (BLE)
+  → Button press on physical device
+  → BLE characteristic notifies with 8-byte payload
+  ↓
+friend-lite-sdk (extras/friend-lite-sdk/)
+  → parse_button_event() converts payload → ButtonState IntEnum
+  ↓
+BLE Client (extras/local-wearable-client/ or mobile app)
+  → Formats as Wyoming protocol: {"type": "button-event", "data": {"state": "SINGLE_TAP"}}
+  → Sends over WebSocket
+  ↓
+Backend (websocket_controller.py)
+  → _handle_button_event() stores marker on client_state
+  → Maps ButtonState → PluginEvent using enums (plugins/events.py)
+  → Dispatches granular event to plugin system
+  ↓
+Plugin System
+  → Routed to subscribed plugins (e.g., test_button_actions)
+  → Plugins use PluginServices for system actions and cross-plugin calls
+```
+
+**Use Cases**:
+- Close current conversation (single press)
+- Toggle smart home devices (double press)
+- Custom actions via cross-plugin communication
+
+**Example**:
+```python
+async def on_button_event(self, context: PluginContext):
+    if context.event == PluginEvent.BUTTON_SINGLE_PRESS:
+        session_id = context.data.get('session_id')
+        await context.services.close_conversation(session_id)
+```
+
+### 5. Plugin Action Events (`plugin_action`)
+
+**When**: Another plugin calls `context.services.call_plugin()`
+**Context Data**:
+- `action` (str): Action name (e.g., `toggle_lights`)
+- Plus any additional data from the calling plugin
+
+**Use Cases**:
+- Cross-plugin communication (button press → toggle lights)
+- Service orchestration between plugins
+
+**Example**:
+```python
+async def on_plugin_action(self, context: PluginContext):
+    action = context.data.get('action')
+    if action == 'toggle_lights':
+        # Handle the action
+        ...
+```
+
+### PluginServices
+
+Plugins receive a `services` object on the context for system and cross-plugin interaction:
+
+```python
+# Close the current conversation (triggers post-processing)
+await context.services.close_conversation(session_id, reason)
+
+# Call another plugin's on_plugin_action() handler
+result = await context.services.call_plugin("homeassistant", "toggle_lights", data)
+```
+
 ## Creating Your First Plugin
 
 ### Step 1: Generate Boilerplate
@@ -225,7 +298,7 @@ import logging
 import re
 from typing import Any, Dict, List, Optional
 
-from ..base import BasePlugin, PluginContext, PluginResult
+from advanced_omi_backend.plugins.base import BasePlugin, PluginContext, PluginResult
 
 logger = logging.getLogger(__name__)
 
@@ -671,7 +744,7 @@ async def on_conversation_complete(self, context):
 **Solution**:
 - Restart backend after adding dependencies
 - Verify imports are from correct modules
-- Check relative imports use `..base` for base classes
+- Use absolute imports for framework classes: `from advanced_omi_backend.plugins.base import BasePlugin`
 
 ### Database Connection Issues
 
@@ -749,12 +822,13 @@ class ExternalServicePlugin(BasePlugin):
 
 ## Resources
 
-- **Base Plugin Class**: `backends/advanced/src/advanced_omi_backend/plugins/base.py`
-- **Example Plugins**:
+- **Plugin Framework**: `backends/advanced/src/advanced_omi_backend/plugins/` (base.py, router.py, events.py, services.py)
+- **Plugin Implementations**: `plugins/` at repo root
   - Email Summarizer: `plugins/email_summarizer/`
   - Home Assistant: `plugins/homeassistant/`
   - Test Event: `plugins/test_event/`
-- **Plugin Generator**: `scripts/create_plugin.py`
+  - Test Button Actions: `plugins/test_button_actions/`
+- **Plugin Generator**: `backends/advanced/scripts/create_plugin.py`
 - **Configuration**: `config/plugins.yml.template`
 
 ## Contributing Plugins

backends/advanced/docker-compose-test.yml

@@ -21,6 +21,7 @@ services:
       - ../../config:/app/config  # Mount config directory with defaults.yml
       - ../../tests/configs:/app/test-configs:ro  # Mount test-specific configs
       - ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml  # Mount test plugins config to correct location
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       # Override with test-specific settings
       - MONGODB_URI=mongodb://mongo-test:27017/test_db
@@ -223,6 +224,7 @@ services:
       - ../../config:/app/config  # Mount config directory with defaults.yml
       - ../../tests/configs:/app/test-configs:ro  # Mount test-specific configs
       - ${PLUGINS_CONFIG:-../../tests/config/plugins.test.yml}:/app/config/plugins.yml  # Mount test plugins config to correct location
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       # Same environment as backend
       - MONGODB_URI=mongodb://mongo-test:27017/test_db

backends/advanced/docker-compose.yml

@@ -40,6 +40,7 @@ services:
       - ./data/debug_dir:/app/debug_dir
       - ./data:/app/data
       - ../../config:/app/config  # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       - DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
       - PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
@@ -95,6 +96,7 @@ services:
       - ./data/audio_chunks:/app/audio_chunks
       - ./data:/app/data
       - ../../config:/app/config  # Mount entire config directory (includes config.yml, defaults.yml, plugins.yml)
+      - ../../plugins:/app/plugins  # External plugins directory
     environment:
       - DEEPGRAM_API_KEY=${DEEPGRAM_API_KEY}
       - PARAKEET_ASR_URL=${PARAKEET_ASR_URL}
@@ -212,8 +214,8 @@ services:
       - "6033:6033" # gRPC
       - "6034:6034" # HTTP
     volumes:
-      - ./data/qdrant_data:/qdrant/storage 
-  
+      - ./data/qdrant_data:/qdrant/storage
+    restart: unless-stopped
 
   mongo:
     image: mongo:8.0.14
@@ -227,6 +229,7 @@ services:
       timeout: 5s
       retries: 5
       start_period: 10s
+    restart: unless-stopped
 
   redis:
     image: redis:7-alpine
@@ -235,6 +238,7 @@ services:
     volumes:
       - ./data/redis_data:/data
     command: redis-server --appendonly yes
+    restart: unless-stopped
     healthcheck:
       test: ["CMD", "redis-cli", "ping"]
       interval: 5s
@@ -267,9 +271,6 @@ services:
       timeout: 10s
       retries: 5
       start_period: 30s
-    profiles:
-      - obsidian
-      - knowledge-graph
 
   # ollama:
   #   image: ollama/ollama:latest