Conversation
- Updated the friend-lite-sdk to version 0.3.0, reflecting the transition to support OMI/Neo1 BLE wearable devices. - Refactored the Bluetooth connection handling to introduce a new `WearableConnection` class, enhancing the connection lifecycle management for wearable devices. - Added a new `Neo1Connection` class for controlling Neo1 devices, including methods for sleep and wake functionalities. - Updated UUID constants to include Neo1-specific characteristics, improving device interaction capabilities. - Revised the plugin development guide to reflect changes in device naming and connection processes. - Removed outdated local OMI Bluetooth scripts and documentation to streamline the project structure and focus on wearable client development.
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR restructures the BLE wearable client from a device-specific implementation (local-omi-bt) to a modular multi-device architecture. Changes introduce a base WearableConnection class with device-specific subclasses (OmiConnection, Neo1Connection), update the SDK exports, remove the old client implementation, and add a new local-wearable-client with macOS menu bar UI, backend streaming, and launchd service management. Changes
Sequence Diagram(s)sequenceDiagram
participant UI as Menu Bar UI
participant AsyncThread as Asyncio Thread
participant BLEMgr as BLE Manager
participant Scanner as BLE Scanner
participant Device as WearableConnection
participant Audio as Audio Pipeline
participant Backend as WebSocket Backend
UI->>AsyncThread: request_scan()
AsyncThread->>BLEMgr: start_scanning()
BLEMgr->>Scanner: perform BLE scan
Scanner-->>BLEMgr: device list
BLEMgr->>UI: update nearby devices
UI->>AsyncThread: request_connect(mac)
AsyncThread->>BLEMgr: _connect(device)
BLEMgr->>Device: establish connection
Device-->>BLEMgr: connected
BLEMgr->>Device: subscribe to audio
Device->>Audio: stream audio chunks
Audio->>Audio: decode opus
Audio->>Audio: write to local sink
Audio->>Backend: forward to backend
Backend-->>Audio: acknowledgement
UI->>AsyncThread: request_disconnect()
AsyncThread->>Device: disconnect
Device-->>BLEMgr: disconnected
BLEMgr->>UI: update status
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes 🚥 Pre-merge checks | ✅ 1 | ❌ 3❌ Failed checks (2 warnings, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…pp functionality - Updated backend_sender.py to stream raw Opus audio instead of PCM, improving bandwidth efficiency. - Modified stream_to_backend function to handle Opus audio data and adjusted audio chunk parameters accordingly. - Enhanced main.py with new CLI commands for device scanning and connection management, improving user experience. - Introduced menu_app.py for a macOS menu bar application, providing a user-friendly interface for device management and status display. - Added README.md to document usage instructions and configuration details for the local wearable client. - Updated requirements.txt to include new dependencies for the menu app and service management. - Implemented service.py for managing launchd service installation and configuration on macOS, enabling auto-start on login.
Collaborator
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Fix all issues with AI agents
In `@extras/local-wearable-client/backend_sender.py`:
- Line 53: The code logs the user's email at INFO via
logger.info("Authenticating with backend as: %s", username), exposing PII;
change this to a non-PII log by either lowering the level to logger.debug or
masking the email before logging (e.g., show only domain or first char + ****),
and ensure the constant ADMIN_EMAIL/variable username is not emitted in full in
authentication flows; update the log call accordingly in backend_sender.py where
logger.info is used.
In `@extras/local-wearable-client/main.py`:
- Around line 205-218: The session deadlocks because asyncio.gather waits for
all tasks (conn.wait_until_disconnected(), process_audio(),
backend_stream_wrapper()) to finish while process_audio() and
backend_stream_wrapper() are infinite loops; change the orchestration to await
the disconnect task first (e.g., use asyncio.wait(...,
return_when=FIRST_COMPLETED) or await the disconnect task directly) and then
cancel the remaining tasks in the tasks list, await their cancellation, and only
after that put None into backend_queue; ensure you call task.cancel() for each
remaining task and handle/await their cancellation so the finally block that
does await backend_queue.put(None) can run and clean up properly.
In `@extras/local-wearable-client/menu_app.py`:
- Around line 200-206: request_disconnect currently only clears _target_mac and
doesn't actually stop an active connection; update it to cancel the background
connect/stream task (store and use _running_task set by _connect when launching
connect_and_stream) and/or schedule a BLE disconnect on the background loop
(call conn.disconnect() or signal the running connect_and_stream coroutine to
exit) so the UI "Disconnect" button immediately terminates the connection;
locate symbols _connect, connect_and_stream, _running_task, conn.disconnect and
ensure request_disconnect cancels _running_task or posts a disconnect to the
background event loop.
- Around line 184-209: The request_connect/_immediate_connect flow can start a
new concurrent _connect while _scan_loop is already connecting, risking
simultaneous connect_and_stream sessions and corrupting shared state like
backend_sender._active_websocket; serialize or cancel overlapping attempts by
adding a single-connection guard. Introduce a connection task handle or
asyncio.Lock on the instance (e.g., self._connect_task or self._connect_lock)
and update request_connect/_immediate_connect and any callers in _scan_loop to:
check if a connect is already running, cancel the existing task (await cancel
and cleanup) or await the lock before starting _connect, assign/clear the handle
on start/finish (including on exceptions) and ensure only one connect_and_stream
runs at a time so backend_sender._active_websocket remains consistent.
In `@extras/local-wearable-client/service.py`:
- Around line 70-74: The plist's ProgramArguments currently passes the "menu"
subcommand which will attempt to start a GUI/menu-bar rumps app under launchd;
update the ProgramArguments array in extras/local-wearable-client/service.py to
use the "run" subcommand instead of "menu" (i.e., the entry that currently
contains "menu" should be replaced with "run") so the launchd agent runs
headless as described in the README; ensure the rest of the ProgramArguments
(uv, "run", "--with-requirements", str(PROJECT_DIR / "requirements.txt"),
"python", str(PROJECT_DIR / "main.py")) are consistent with how your main.py
handles subcommands.
- Around line 60-66: The manual .env parser reads lines into env but doesn't
remove surrounding quotes from values (so KEY="value" becomes value with
quotes); update the parsing loop around env_file.read_text().splitlines() to
strip surrounding single or double quotes from the parsed value before assigning
into env (handle both value starting and ending with the same quote char and
remove them), and ensure you still trim whitespace via value.strip() and
preserve empty/unquoted values; modify the block that produces key, _, value =
line.partition("=") (and the subsequent env[key.strip()] = value.strip()) to
perform this quote-stripping step.
🧹 Nitpick comments (10)
extras/local-wearable-client/requirements.txt (1)
5-7: No version pins on new dependencies.
websockets,pyyaml, andrumpslack version constraints. While the SDK'spyproject.tomlpins minimum versions for its deps, these top-level requirements could break on a major bump. Consider adding minimum version pins (e.g.,pyyaml>=6.0,rumps>=0.4.0) for reproducibility.extras/local-wearable-client/service.py (1)
159-172:logs()withfollow=Falsereads entire file into memory.
LOG_FILE.read_text()[-5000:]loads the whole log file before slicing. For large log files this is wasteful. Consider usingtailfor the non-follow case too.Proposed fix
else: - print(LOG_FILE.read_text()[-5000:]) # Last 5000 chars + subprocess.run(["tail", "-n", "100", str(LOG_FILE)])extras/friend-lite-sdk/friend_lite/neo1.py (1)
8-16: Consider extracting the "not connected" guard into the base class.The
if self._client is None: raise RuntimeError(...)pattern is repeated here in bothsleep()andwake(), and also exists inWearableConnection.subscribe()(line 58–59 ofbluetooth.py). A small helper on the base class would DRY this up and ensure a consistent error message everywhere.♻️ Suggested helper in WearableConnection
Add to
WearableConnectioninbluetooth.py:def _require_connected(self) -> BleakClient: """Return the active BleakClient or raise if not connected.""" if self._client is None: raise RuntimeError("Not connected to device") return self._clientThen in
neo1.py:async def sleep(self) -> None: - if self._client is None: - raise RuntimeError("Not connected to device") - await self._client.write_gatt_char(NEO1_CTRL_CHAR_UUID, b"\x00", response=True) + client = self._require_connected() + await client.write_gatt_char(NEO1_CTRL_CHAR_UUID, b"\x00", response=True) async def wake(self) -> None: - if self._client is None: - raise RuntimeError("Not connected to device") - await self._client.write_gatt_char(NEO1_CTRL_CHAR_UUID, b"\x01", response=True) + client = self._require_connected() + await client.write_gatt_char(NEO1_CTRL_CHAR_UUID, b"\x01", response=True)extras/local-wearable-client/backend_sender.py (2)
92-92: Parameterloggershadows the module-levellogger.The
loggerparameter onreceive_handlershadows the module-levelloggerdefined on line 29. If this function is always called with the module logger (as on line 157), the parameter is redundant and adds confusion. Consider removing it and using the module-level logger directly.♻️ Proposed fix
-async def receive_handler(websocket, logger) -> None: +async def receive_handler(websocket) -> None:And update the call site at line 157:
- receive_task = asyncio.create_task(receive_handler(websocket, logger)) + receive_task = asyncio.create_task(receive_handler(websocket))
142-152:_active_websocketglobal is not safe for concurrent streams.If
stream_to_backendis called concurrently (e.g., two devices streaming), theglobal _active_websocketassignment on line 152 will silently overwrite the previous reference, and thefinallyblock (line 201) will clear it, breaking button events for the other stream. This is acceptable now if there's only ever one connection, but worth noting with a comment or an assertion to guard against future misuse.extras/local-wearable-client/menu_app.py (2)
65-70: Busy-wait creates a newthreading.Eventon each iteration just to sleep.
threading.Event().wait(0.01)allocates a new event object every 10ms. Usetime.sleep(0.01)instead for a simple delay.♻️ Proposed fix
+import time ... def start(self) -> None: self._thread = threading.Thread(target=self._run, daemon=True) self._thread.start() # Wait until the loop is running while self.loop is None or not self.loop.is_running(): - threading.Event().wait(0.01) + time.sleep(0.01)
50-53:update()accepts arbitrary kwargs without validation.A typo like
state.update(staus="connected")will silently create a new attribute instead of updatingstatus. Consider validating keys against known fields.♻️ Suggested validation
def update(self, **kwargs) -> None: with self._lock: for k, v in kwargs.items(): + if not hasattr(self, k): + raise AttributeError(f"SharedState has no field '{k}'") setattr(self, k, v)extras/local-wearable-client/main.py (3)
129-142:backend_queueis referenced in this closure before it's defined on line 163.
handle_ble_datacapturesbackend_queueon line 140, but the variable is assigned 23 lines later. This works at runtime because the callback is only invoked aftersubscribe_audio(line 196), well after line 163 executes. However, this is fragile and confusing to readers. Consider movingbackend_queueinitialization above the callback definition.♻️ Suggested reordering
+ # Cap at 500 chunks so a dead backend doesn't eat memory + backend_queue: asyncio.Queue[bytes | None] = asyncio.Queue(maxsize=500) + def handle_ble_data(_sender: Any, data: bytes) -> None: ... - - # Cap at 500 chunks so a dead backend doesn't eat memory - backend_queue: asyncio.Queue[bytes | None] = asyncio.Queue(maxsize=500)
152-156: Fire-and-forget task: exception fromsend_button_eventwill be silently swallowed.
loop.create_task(...)without storing the reference means any exception raised bysend_button_eventis lost (Python emits a warning to stderr but it's easy to miss). Store the task reference or add a done callback to log errors.♻️ Example with error callback
- loop.create_task(send_button_event(state.name)) + task = loop.create_task(send_button_event(state.name)) + task.add_done_callback( + lambda t: t.exception() and logger.error("Button event error: %s", t.exception()) + )
89-89: Missing return type annotation onscan_for_device.The function returns
Optional[dict]but has no return type annotation. Other functions in this file are annotated.♻️ Suggested fix
-async def scan_for_device(config: dict): +async def scan_for_device(config: dict) -> dict | None:
- Removed the audio queue in favor of a dedicated BLE data queue and backend queue for improved data handling. - Enhanced the `connect_and_stream` function to streamline audio decoding and writing to the local file sink. - Updated the handling of BLE data to ensure robust queue management and error logging. - Improved task management during device disconnection to ensure proper cleanup and error handling. - Updated requirements.txt to specify a minimum version for easy_audio_interfaces, ensuring compatibility.
AnkushMalaker
added a commit
that referenced
this pull request
Feb 15, 2026
* Enhance ASR service descriptions and provider feedback in wizard.py - Updated the description for the 'asr-services' to remove the specific mention of 'Parakeet', making it more general. - Improved the console output for auto-selected services to include the transcription provider label, enhancing user feedback during service selection. * Implement LangFuse integration for observability and prompt management - Added LangFuse configuration options in the .env.template for observability and prompt management. - Introduced setup_langfuse method in ChronicleSetup to handle LangFuse initialization and configuration prompts. - Enhanced prompt management by integrating a centralized PromptRegistry for dynamic prompt retrieval and registration. - Updated various services to utilize prompts from the PromptRegistry, improving flexibility and maintainability. - Refactored OpenAI client initialization to support optional LangFuse tracing, enhancing observability during API interactions. - Added new prompt defaults for memory management and conversation handling, ensuring consistent behavior across the application. * Enhance LangFuse integration and service management - Added LangFuse service configuration in services.py and wizard.py, including paths, commands, and descriptions. - Implemented auto-selection for LangFuse during service setup, improving user experience. - Enhanced service startup process to display prompt management tips for LangFuse, guiding users on editing AI prompts. - Updated run_service_setup to handle LangFuse-specific parameters, including admin credentials and API keys, ensuring seamless integration with backend services. * Feat/better reprocess memory (#300) * Enhance ASR service descriptions and provider feedback in wizard.py (#290) - Updated the description for the 'asr-services' to remove the specific mention of 'Parakeet', making it more general. - Improved the console output for auto-selected services to include the transcription provider label, enhancing user feedback during service selection. * Refactor Obsidian and Knowledge Graph integration in services and setup - Removed redundant Obsidian and Knowledge Graph configuration checks from services.py, streamlining the command execution process. - Updated wizard.py to enhance user experience by setting default options for speaker recognition during service selection. - Improved Neo4j password handling in setup processes, ensuring consistent configuration prompts and feedback. - Introduced a new cron scheduler for managing scheduled tasks, enhancing the backend's automation capabilities. - Added new entity annotation features, allowing for corrections and updates to knowledge graph entities directly through the API. * Enhance ASR services configuration and VibeVoice integration - Added new configuration options for VibeVoice ASR in defaults.yml, including batching parameters for audio processing. - Updated Docker Compose files to mount the config directory, ensuring access to ASR service configurations. - Enhanced the VibeVoice transcriber to load configuration settings from defaults.yml, allowing for dynamic adjustments via environment variables. - Introduced quantization options for model loading in the VibeVoice transcriber, improving performance and flexibility. - Refactored the speaker identification process to streamline audio handling and improve logging for better debugging. - Updated documentation to reflect new configuration capabilities and usage instructions for the VibeVoice ASR provider. * Enhance LangFuse integration and memory reprocessing capabilities - Introduced functions for checking LangFuse configuration in services.py, ensuring proper setup for observability. - Updated wizard.py to facilitate user input for LangFuse configuration, including options for local and external setups. - Implemented memory reprocessing logic in memory services to update existing memories based on speaker re-identification. - Enhanced speaker recognition client to support per-segment identification, improving accuracy during reprocessing. - Refactored various components to streamline handling of LangFuse parameters and improve overall service management. * Enhance service management and user input handling - Updated services.py to include LangFuse configuration checks during service startup, improving observability setup. - Refactored wizard.py to utilize a masked input for Neo4j password prompts, enhancing user experience and security. - Improved cron scheduler in advanced_omi_backend to manage active tasks and validate cron expressions, ensuring robust job execution. - Enhanced speaker recognition client documentation to clarify user_id limitations, preparing for future multi-user support. - Updated knowledge graph routes to enforce validation on entity updates, ensuring at least one field is provided for updates. * fix: Plugin System Refactor (#301) * Refactor connect-omi.py for improved device selection and user interaction - Replaced references to the chronicle Bluetooth library with friend_lite for device management. - Removed the list_devices function and implemented a new prompt_user_to_pick_device function to enhance user interaction when selecting OMI/Neo devices. - Updated the find_and_set_omi_mac function to utilize the new device selection method, improving the overall flow of device connection. - Added a new scan_devices.py script for quick scanning of neo/neosapien devices, enhancing usability. - Updated README.md to reflect new usage instructions and prerequisites for connecting to OMI devices over Bluetooth. - Enhanced start.sh to ensure proper environment variable setup for macOS users. * Add friend-lite-sdk: Initial implementation of Python SDK for OMI/Friend Lite BLE devices - Introduced the friend-lite-sdk, a Python SDK for OMI/Friend Lite BLE devices, enabling audio streaming, button events, and transcription functionalities. - Added LICENSE and NOTICE files to clarify licensing and attribution. - Created pyproject.toml for package management, specifying dependencies and project metadata. - Developed core modules including bluetooth connection handling, button event parsing, audio decoding, and transcription capabilities. - Implemented example usage in README.md to guide users on installation and basic functionality. - Enhanced connect-omi.py to utilize the new SDK for improved device management and event handling. - Updated requirements.txt to reference the new SDK for local development. This commit lays the foundation for further enhancements and integrations with OMI devices. * Enhance client state and plugin architecture for button event handling - Introduced a new `markers` list in `ClientState` to collect button event data during sessions. - Added `add_marker` method to facilitate the addition of markers to the current session. - Implemented `on_button_event` method in the `BasePlugin` class to handle device button events, providing context data for button state and timestamps. - Updated `PluginRouter` to route button events to the appropriate plugin handler. - Enhanced conversation job handling to attach markers from Redis sessions, improving the tracking of button events during conversations. * Move plugins locatino - Introduced the Email Summarizer plugin that automatically sends email summaries upon conversation completion. - Implemented SMTP email service for sending formatted HTML and plain text emails. - Added configuration options for SMTP settings and email content in `config.yml`. - Created setup script for easy configuration of SMTP credentials and plugin orchestration. - Enhanced documentation with usage instructions and troubleshooting tips for the plugin. - Updated existing plugin architecture to support new event handling for email summaries. * Enhance Docker Compose and Plugin Management - Added external plugins directory to Docker Compose files for better plugin management. - Updated environment variables for MongoDB and Redis services to ensure consistent behavior. - Introduced new dependencies in `uv.lock` for improved functionality. - Refactored audio processing to support various audio formats and enhance error handling. - Implemented new plugin event types and services for better integration and communication between plugins. - Enhanced conversation and session management to support new closing mechanisms and event logging. * Update audio processing and event logging - Increased the maximum event log size in PluginRouter from 200 to 1000 for improved event tracking. - Refactored audio stream producer to dynamically read audio format from Redis session metadata, enhancing flexibility in audio handling. - Updated transcription job processing to utilize session-specific audio format settings, ensuring accurate audio processing. - Enhanced audio file writing utility to accept PCM parameters, allowing for better control over audio data handling. * Add markers list to ClientState and update timeout trigger comment - Introduced a new `markers` list in `ClientState` to track button event data during conversations. - Updated comment in `open_conversation_job` to clarify the behavior of the `timeout_triggered` variable, ensuring better understanding of session management. * Refactor audio file logging and error handling - Updated audio processing logs to consistently use the `filename` variable instead of `file.filename` for clarity. - Enhanced error logging to utilize the `filename` variable, improving traceability of issues during audio processing. - Adjusted title generation logic to handle cases where the filename is "unknown," ensuring a default title is used. - Minor refactor in conversation closing logs to use `user.user_id` for better consistency in user identification. * Enhance conversation retrieval with pagination and orphan handling - Updated `get_conversations` function to support pagination through `limit` and `offset` parameters, improving performance for large datasets. - Consolidated query logic to fetch both normal and orphan conversations in a single database call, reducing round-trips and enhancing efficiency. - Modified the response structure to include total count, limit, and offset in the returned data for better client-side handling. - Adjusted database indexing to optimize queries for paginated results, ensuring faster access to conversation data. * Refactor connection logging in transcribe function - Moved connection logging for the Wyoming server to a more structured format within the `transcribe_wyoming` function. - Ensured that connection attempts and successes are logged consistently for better traceability during audio transcription processes. * Feat/neo sdk (#302) * Update friend-lite-sdk for Neo1 device support and enhance documentation - Updated the friend-lite-sdk to version 0.3.0, reflecting the transition to support OMI/Neo1 BLE wearable devices. - Refactored the Bluetooth connection handling to introduce a new `WearableConnection` class, enhancing the connection lifecycle management for wearable devices. - Added a new `Neo1Connection` class for controlling Neo1 devices, including methods for sleep and wake functionalities. - Updated UUID constants to include Neo1-specific characteristics, improving device interaction capabilities. - Revised the plugin development guide to reflect changes in device naming and connection processes. - Removed outdated local OMI Bluetooth scripts and documentation to streamline the project structure and focus on wearable client development. * Refactor backend audio streaming to use Opus codec and enhance menu app functionality - Updated backend_sender.py to stream raw Opus audio instead of PCM, improving bandwidth efficiency. - Modified stream_to_backend function to handle Opus audio data and adjusted audio chunk parameters accordingly. - Enhanced main.py with new CLI commands for device scanning and connection management, improving user experience. - Introduced menu_app.py for a macOS menu bar application, providing a user-friendly interface for device management and status display. - Added README.md to document usage instructions and configuration details for the local wearable client. - Updated requirements.txt to include new dependencies for the menu app and service management. - Implemented service.py for managing launchd service installation and configuration on macOS, enabling auto-start on login. * Refactor audio processing and queue management in local wearable client - Removed the audio queue in favor of a dedicated BLE data queue and backend queue for improved data handling. - Enhanced the `connect_and_stream` function to streamline audio decoding and writing to the local file sink. - Updated the handling of BLE data to ensure robust queue management and error logging. - Improved task management during device disconnection to ensure proper cleanup and error handling. - Updated requirements.txt to specify a minimum version for easy_audio_interfaces, ensuring compatibility.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
Release Notes
New Features
Documentation
Refactor