diff --git a/Docs/ssl-certificates.md b/Docs/ssl-certificates.md new file mode 100644 index 00000000..1980c833 --- /dev/null +++ b/Docs/ssl-certificates.md @@ -0,0 +1,73 @@ +# SSL Certificates & HTTPS + +Chronicle uses automatic HTTPS setup for secure microphone access and remote connections. + +## Why HTTPS is Needed + +Modern browsers require HTTPS for: +- **Microphone access** over network (not localhost) +- **Secure WebSocket connections** (WSS) +- **Remote access** via Tailscale/VPN +- **Production deployments** + +## SSL Implementation + +### Advanced Backend → Caddy + +The main backend uses **Caddy** for automatic HTTPS: + +**Configuration**: `backends/advanced/Caddyfile` +**Activation**: Caddy starts when using `--profile https` or when wizard enables HTTPS +**Certificate**: Self-signed for local/Tailscale IPs, automatic Let's Encrypt for domains + +**Ports**: +- `443` - HTTPS (main access) +- `80` - HTTP (redirects to HTTPS) + +**Access**: `https://localhost` or `https://your-tailscale-ip` + +### Speaker Recognition → nginx + +The speaker recognition service uses **nginx** for HTTPS: + +**Configuration**: `extras/speaker-recognition/nginx.conf` +**Certificate**: Self-signed via `ssl/generate-ssl.sh` + +**Ports**: +- `8444` - HTTPS +- `8081` - HTTP (redirects to HTTPS) + +**Access**: `https://localhost:8444` + +## Setup via Wizard + +When you run `./wizard.sh`, the setup wizard: +1. Asks if you want to enable HTTPS +2. Prompts for your Tailscale IP or domain +3. Generates SSL certificates automatically +4. Configures Caddy/nginx as needed +5. Updates CORS settings for HTTPS origins + +**No manual setup required** - the wizard handles everything. + +## Browser Certificate Warnings + +Since we use self-signed certificates for local/Tailscale IPs, browsers will show security warnings: + +1. Click "Advanced" +2. Click "Proceed to localhost (unsafe)" or similar +3. Microphone access will now work + +For production with real domains, Caddy automatically obtains valid Let's Encrypt certificates. + +## Troubleshooting + +**HTTPS not working**: +- Check Caddy/nginx containers are running: `docker compose ps` +- Verify certificates exist: `ls backends/advanced/ssl/` or `ls extras/speaker-recognition/ssl/` +- Check you're using `https://` not `http://` + +**Microphone not accessible**: +- Ensure you're accessing via HTTPS (not HTTP) +- Accept browser certificate warning +- Verify you're not using `localhost` from remote device (use Tailscale IP instead) diff --git a/backends/advanced/Docs/HTTPS_SETUP.md b/backends/advanced/Docs/HTTPS_SETUP.md deleted file mode 100644 index 54852a20..00000000 --- a/backends/advanced/Docs/HTTPS_SETUP.md +++ /dev/null @@ -1,255 +0,0 @@ -# HTTPS Setup for Chronicle Advanced Backend - -This guide explains how to set up HTTPS/SSL access for Chronicle Advanced Backend, enabling secure microphone access and network connectivity. - -## Why HTTPS is Needed - -Modern browsers require HTTPS for: -- **Microphone access** over network connections (not localhost) -- **Secure WebSocket connections** (WSS) -- **Tailscale/VPN access** with audio features -- **Production deployments** - -## Quick Setup - -### 1. Initialize HTTPS with Your IP - -Run the initialization script with your Tailscale or network IP: - -```bash -cd backends/advanced -./init.sh 100.83.66.30 # Replace with your actual IP -``` - -This script will: -- Generate SSL certificates for localhost and your IP -- Create nginx configuration files -- Update CORS settings for HTTPS origins - -### 2. Start with HTTPS Proxy - -```bash -# HTTPS with nginx proxy (REQUIRED for network microphone access) -docker compose up --build -d - -# HTTP only (no nginx, localhost microphone access only) -docker compose up --build -d -``` - -**NOTE**: The nginx service now starts automatically with the standard docker compose command, providing immediate HTTPS access when SSL certificates are configured. - -### 3. Access the Services - -#### Chronicle Advanced Backend (Primary - ports 80/443) -- **HTTPS:** https://localhost/ or https://your-ip/ (accept SSL certificate) -- **HTTP:** http://localhost/ (redirects to HTTPS) -- **Features:** Dashboard, Live Recording, Conversations, Memories - -#### Speaker Recognition Service (Secondary - ports 8081/8444) -- **HTTPS:** https://localhost:8444/ or https://your-ip:8444/ (accept SSL certificate) -- **HTTP:** http://localhost:8081/ (redirects to HTTPS) -- **Features:** Speaker enrollment, audio analysis, live inference - -## Port Allocation - -### Advanced Backend (Primary Service) -- **Port 80:** HTTP (redirects to HTTPS) -- **Port 443:** HTTPS with nginx proxy -- **Port 5173:** Direct Vite dev server (development only) -- **Port 8000:** Direct backend API (development only) - -### Speaker Recognition (Secondary Service) -- **Port 8081:** HTTP (redirects to HTTPS) -- **Port 8444:** HTTPS with nginx proxy -- **Port 5175:** Direct React dev server (internal) -- **Port 8085:** Direct API service (internal) - -## Manual Setup - -### SSL Certificate Generation - -If you need to regenerate certificates: - -```bash -cd ssl -./generate-ssl.sh 100.83.66.30 # Your IP address -``` - -### Environment Configuration - -Update your `.env` file to include HTTPS origins: - -```bash -CORS_ORIGINS=https://localhost,https://127.0.0.1,https://100.83.66.30 -``` - -## Docker Compose Profiles - -### With HTTPS Configuration (Network Access) -**Services started:** -- ✅ nginx (ports 443/80) - SSL termination and proxy -- ✅ webui (port 5173, internal) - Vite dev server -- ✅ chronicle-backend (port 8000, internal) -- ✅ mongo, qdrant (databases) - -**Access:** https://localhost/ or https://your-ip/ -**Microphone:** Works over network with HTTPS - -### Without HTTPS Configuration (Default - Localhost Only) -**Services started:** -- ✅ nginx (ports 443/80) - but without SSL certificates -- ✅ webui (port 5173, direct access) - Vite dev server -- ✅ chronicle-backend (port 8000) -- ✅ mongo, qdrant (databases) - -**Access:** http://localhost:5173 -**Microphone:** Only works on localhost (browser security) - -## Nginx Configuration - -The setup uses a single nginx configuration: - -### Single Config (`nginx.conf.template`) -- Proxies to `webui:5173` for the Vite dev server -- Handles WebSocket connections for audio streaming -- SSL termination with proper headers -- Supports Vite HMR (Hot Module Replacement) over WSS -- Always provides development experience with hot reload - -## WebSocket Endpoints - -All WebSocket endpoints are proxied through nginx with SSL: - -- **`wss://your-ip/ws_pcm`** - Primary audio streaming (Wyoming protocol + PCM) -- **`wss://your-ip/ws_omi`** - OMI device audio streaming (Wyoming protocol + Opus) -- **`wss://your-ip/ws`** - Legacy audio streaming (Opus packets) - -**Note:** When accessed through HTTPS proxy, all API calls use relative URLs automatically. - -## Browser Certificate Trust - -Since we use self-signed certificates, browsers will show security warnings: - -### Chrome/Edge -1. Visit https://localhost/ -2. Click "Advanced" → "Proceed to localhost (unsafe)" -3. Or add certificate to trusted store - -### Firefox -1. Visit https://localhost/ -2. Click "Advanced" → "Accept the Risk and Continue" - -### Safari -1. Visit https://localhost/ -2. Click "Show Details" → "visit this website" - -## Troubleshooting - -### Certificate Issues - -**Problem:** "SSL certificate problem: self signed certificate" -**Solution:** -```bash -# Regenerate certificates -cd ssl -./generate-ssl.sh your-ip -docker compose restart nginx -``` - -### WebSocket Connection Fails - -**Problem:** WSS connection refused -**Solution:** -1. Check nginx is running: `docker compose ps nginx` -2. Verify certificate: `curl -k https://localhost/health` -3. Check logs: `docker compose logs nginx` - -### CORS Errors - -**Problem:** "Cross-Origin Request Blocked" -**Solution:** -1. Update CORS_ORIGINS in `.env` to include your HTTPS origin -2. Restart backend: `docker compose restart chronicle-backend` - -### Microphone Access Denied - -**Problem:** Browser blocks microphone access -**Solution:** -1. Ensure you're using HTTPS (not HTTP) -2. Accept SSL certificate warnings -3. Grant microphone permissions when prompted - -## Port Reference - -### HTTPS Setup (Production) -- **443** - HTTPS (nginx → webui:80) -- **80** - HTTP redirect to HTTPS - -### HTTPS Setup (Development) -- **8443** - HTTPS (nginx-dev → webui-dev:5173) -- **8080** - HTTP redirect to HTTPS - -### Standard Setup -- **3000** - HTTP (webui production) -- **5173** - HTTP (webui development) -- **8000** - HTTP (chronicle-backend) - -## Live Recording Feature - -The Live Recording feature automatically adapts to your connection: - -- **HTTP + localhost:** Uses `ws://localhost:8000/ws_pcm` -- **HTTPS:** Uses `wss://your-domain/ws_pcm` -- **Microphone access:** Requires HTTPS for network connections - -Access at: -- Local: https://localhost/live-record -- Network: https://your-ip/live-record - -## Security Considerations - -### Self-Signed Certificates -- Only for development and local network use -- Use proper CA certificates for production -- Consider Let's Encrypt for public deployments - -### Network Security -- HTTPS encrypts all traffic including WebSocket data -- Nginx handles SSL termination -- Backend services remain on internal Docker network - -### Browser Security -- Modern browsers block microphone access over HTTP (except localhost) -- WSS required for secure WebSocket connections over network -- CORS properly configured for cross-origin requests - -## Production Deployment - -For production deployments: - -1. **Use proper SSL certificates** (Let's Encrypt, commercial CA) -2. **Update nginx configuration** with your domain name -3. **Configure DNS** to point to your server -4. **Use production docker compose profile**: - ```bash - docker compose up -d - ``` - -## Integration with Other Services - -### Speaker Recognition -If using the speaker recognition service alongside Chronicle: - -```bash -# Use different HTTPS ports to avoid conflicts -# Speaker Recognition: 443/80 -# Chronicle: 8443/8080 -docker compose up -d -``` - -### Tailscale Integration -The setup is optimized for Tailscale usage: - -- SSL certificates include your Tailscale IP -- CORS automatically supports 100.x.x.x IP range -- WebSocket connections work over Tailscale network \ No newline at end of file diff --git a/backends/advanced/Docs/UI.md b/backends/advanced/Docs/UI.md index 6447a2a0..02bdf943 100644 --- a/backends/advanced/Docs/UI.md +++ b/backends/advanced/Docs/UI.md @@ -10,7 +10,7 @@ The Chronicle web dashboard provides a comprehensive interface for managing conv ### Dashboard URL - **HTTP**: `http://localhost:5173` (development) or `http://localhost:3000` (production) -- **HTTPS**: `https://localhost/` (with HTTPS configuration via `init-https.sh`) +- **HTTPS**: `https://localhost/` (automatic via setup wizard - see [Docs/ssl-certificates.md](../../../Docs/ssl-certificates.md)) - **Live Recording**: Available at `/live-record` page for real-time audio streaming - **Network Access**: Configure `BACKEND_PUBLIC_URL` for remote device access via Tailscale/LAN diff --git a/backends/advanced/README.md b/backends/advanced/README.md index d493241c..0f5a4490 100644 --- a/backends/advanced/README.md +++ b/backends/advanced/README.md @@ -34,15 +34,10 @@ Modern React-based web dashboard located in `./webui/` with: - **Transcription Provider**: Choose between Deepgram, Mistral, or Offline (Parakeet) - **LLM Provider**: Choose between OpenAI (recommended) or Ollama for memory extraction - **Memory Provider**: Choose between Friend-Lite Native or OpenMemory MCP +- **HTTPS Configuration**: Optional SSL setup for microphone access (uses Caddy) - **Optional Services**: Speaker Recognition, network configuration - **API Keys**: Prompts for all required keys with helpful links -**HTTPS Setup (Optional):** -```bash -# For microphone access and secure connections -./setup-https.sh your-tailscale-ip -``` - #### 2. Start Services **HTTP Mode (Default - No SSL required):** @@ -55,25 +50,13 @@ docker compose up --build -d **HTTPS Mode (For network access and microphone features):** ```bash -# Start with nginx SSL proxy - requires SSL setup first (see below) -docker compose up --build -d +# Start with HTTPS (requires Caddy configuration from wizard) +docker compose --profile https up --build -d ``` - **Web Dashboard**: https://localhost/ or https://your-ip/ - **Backend API**: https://localhost/api/ or https://your-ip/api/ -#### 3. HTTPS Setup (Optional - For Network Access & Microphone Features) - -For network access and microphone features, HTTPS can be configured during initialization or separately: - -```bash -# If not done during init.sh, run HTTPS setup -./init-https.sh 100.83.66.30 # Replace with your IP - -# Start with HTTPS proxy -docker compose up --build -d -``` - -#### Access URLs +#### 3. Access URLs **Friend-Lite Advanced Backend (Primary - ports 80/443):** - **HTTPS Dashboard**: https://localhost/ or https://your-ip/ @@ -91,7 +74,7 @@ docker compose up --build -d - 🌐 **Network Access** from other devices via Tailscale/LAN - 🔄 **Automatic protocol detection** - Frontend auto-configures for HTTP/HTTPS -See [Docs/HTTPS_SETUP.md](Docs/HTTPS_SETUP.md) for detailed configuration. +See [Docs/ssl-certificates.md](../../Docs/ssl-certificates.md) for how SSL is configured. ## Testing diff --git a/backends/advanced/init-https.sh b/backends/advanced/init-https.sh deleted file mode 100755 index d1c1b5af..00000000 --- a/backends/advanced/init-https.sh +++ /dev/null @@ -1,96 +0,0 @@ -#!/bin/bash -set -e - -# Initialize Chronicle Advanced Backend with HTTPS proxy -# Usage: ./init.sh - -if [ $# -ne 1 ]; then - echo "Usage: $0 " - echo "Example: $0 100.83.66.30" - echo "" - echo "This script will:" - echo " 1. Generate SSL certificates for localhost and your Tailscale IP" - echo " 2. Create nginx.conf from template" - echo " 3. Set up HTTPS proxy for the backend" - exit 1 -fi - -TAILSCALE_IP="$1" - -# Validate IP format (basic check) -if ! echo "$TAILSCALE_IP" | grep -E '^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$' > /dev/null; then - echo "Error: Invalid IP format. Expected format: xxx.xxx.xxx.xxx" - exit 1 -fi - -echo "🚀 Initializing Chronicle Advanced Backend with Tailscale IP: $TAILSCALE_IP" -echo "" - -# Check if nginx.conf.template exists -if [ ! -f "nginx.conf.template" ]; then - echo "❌ Error: nginx.conf.template not found" - echo " Make sure you're running this from the backends/advanced directory" - exit 1 -fi - -# Generate SSL certificates -echo "📄 Step 1: Generating SSL certificates..." -if [ -f "ssl/generate-ssl.sh" ]; then - ./ssl/generate-ssl.sh "$TAILSCALE_IP" - echo "✅ SSL certificates generated" -else - echo "❌ Error: ssl/generate-ssl.sh not found" - exit 1 -fi - -echo "" - -# Create nginx.conf from template -echo "📄 Step 2: Creating nginx configuration..." -sed "s/TAILSCALE_IP/$TAILSCALE_IP/g" nginx.conf.template > nginx.conf -echo "✅ nginx.conf created with IP: $TAILSCALE_IP" - -echo "" - -# Update .env file with HTTPS CORS origins -echo "📄 Step 3: Updating CORS origins..." -if [ -f ".env" ]; then - # Update existing .env file - if grep -q "CORS_ORIGINS" .env; then - # Update existing CORS_ORIGINS line - sed -i "s/CORS_ORIGINS=.*/CORS_ORIGINS=https:\/\/localhost,https:\/\/localhost:443,https:\/\/127.0.0.1,https:\/\/$TAILSCALE_IP/" .env - else - # Add CORS_ORIGINS line - echo "CORS_ORIGINS=https://localhost,https://localhost:443,https://127.0.0.1,https://$TAILSCALE_IP" >> .env - fi - echo "✅ Updated CORS origins in .env file" -else - echo "⚠️ No .env file found. You may need to:" - echo " 1. Copy .env.template to .env" - echo " 2. Add: CORS_ORIGINS=https://localhost,https://localhost:443,https://127.0.0.1,https://$TAILSCALE_IP" -fi - -echo "" -echo "📄 Step 4: Memory configuration now lives in config.yml (memory section)" - -echo "" -echo "🎉 Initialization complete!" -echo "" -echo "Next steps:" -echo " 1. Start the services:" -echo " docker compose up --build -d" -echo "" -echo " 2. Access the dashboard:" -echo " 🌐 https://localhost/ (accept SSL certificate)" -echo " 🌐 https://$TAILSCALE_IP/" -echo "" -echo " 3. Test live recording:" -echo " 📱 Navigate to Live Record page" -echo " 🎤 Microphone access will work over HTTPS" -echo "" -echo "🔧 Services included:" -echo " - Chronicle Backend: Internal (proxied through nginx)" -echo " - Web Dashboard: https://localhost/ or https://$TAILSCALE_IP/" -echo " - WebSocket Audio: wss://localhost/ws_pcm or wss://$TAILSCALE_IP/ws_pcm" -echo "" -echo "📚 For more details, see: Docs/HTTPS_SETUP.md" diff --git a/backends/advanced/init.py b/backends/advanced/init.py index fe04fd15..dddbfdcb 100644 --- a/backends/advanced/init.py +++ b/backends/advanced/init.py @@ -7,6 +7,7 @@ import argparse import getpass import os +import platform import secrets import shutil import subprocess @@ -15,7 +16,6 @@ from pathlib import Path from typing import Any, Dict -import yaml from dotenv import get_key, set_key from rich.console import Console from rich.panel import Panel @@ -157,13 +157,36 @@ def setup_transcription(self): self.console.print("[blue][INFO][/blue] API keys are stored in .env") self.console.print() - choices = { - "1": "Deepgram (recommended - high quality, cloud-based)", - "2": "Offline (Parakeet ASR - requires GPU, runs locally)", - "3": "None (skip transcription setup)" - } + # Check if transcription provider was provided via command line + if hasattr(self.args, 'transcription_provider') and self.args.transcription_provider: + provider = self.args.transcription_provider + self.console.print(f"[green][SUCCESS][/green] Transcription provider configured via wizard: {provider}") + + # Map provider to choice + if provider == "deepgram": + choice = "1" + elif provider == "parakeet": + choice = "2" + elif provider == "none": + choice = "3" + else: + choice = "1" # Default to Deepgram + else: + # Interactive prompt + is_macos = platform.system() == 'Darwin' + + if is_macos: + parakeet_desc = "Offline (Parakeet ASR - CPU-based, runs locally)" + else: + parakeet_desc = "Offline (Parakeet ASR - GPU recommended, runs locally)" - choice = self.prompt_choice("Choose your transcription provider:", choices, "1") + choices = { + "1": "Deepgram (recommended - high quality, cloud-based)", + "2": parakeet_desc, + "3": "None (skip transcription setup)" + } + + choice = self.prompt_choice("Choose your transcription provider:", choices, "1") if choice == "1": self.console.print("[blue][INFO][/blue] Deepgram selected") @@ -446,30 +469,7 @@ def setup_https(self): except subprocess.CalledProcessError: self.console.print("[yellow][WARNING][/yellow] SSL certificate generation failed") else: - self.console.print(f"[yellow][WARNING][/yellow] SSL script not found at {ssl_script}") - - # Generate nginx.conf from template - self.console.print("[blue][INFO][/blue] Creating nginx configuration...") - nginx_template = script_dir / "nginx.conf.template" - if nginx_template.exists(): - try: - with open(nginx_template, 'r') as f: - nginx_content = f.read() - - # Replace TAILSCALE_IP with server_ip - nginx_content = nginx_content.replace('TAILSCALE_IP', server_ip) - - with open('nginx.conf', 'w') as f: - f.write(nginx_content) - - self.console.print(f"[green][SUCCESS][/green] nginx.conf created for: {server_ip}") - self.config["HTTPS_ENABLED"] = "true" - self.config["SERVER_IP"] = server_ip - - except Exception as e: - self.console.print(f"[yellow][WARNING][/yellow] nginx.conf generation failed: {e}") - else: - self.console.print("[yellow][WARNING][/yellow] nginx.conf.template not found") + self.console.print(f"[yellow][WARNING][/warning] SSL script not found at {ssl_script}") # Generate Caddyfile from template self.console.print("[blue][INFO][/blue] Creating Caddyfile configuration...") @@ -496,6 +496,8 @@ def setup_https(self): f.write(caddyfile_content) self.console.print(f"[green][SUCCESS][/green] Caddyfile created for: {server_ip}") + self.config["HTTPS_ENABLED"] = "true" + self.config["SERVER_IP"] = server_ip except Exception as e: self.console.print(f"[red]❌ ERROR: Caddyfile generation failed: {e}[/red]") @@ -690,10 +692,13 @@ def run(self): def main(): """Main entry point""" parser = argparse.ArgumentParser(description="Chronicle Advanced Backend Setup") - parser.add_argument("--speaker-service-url", + parser.add_argument("--speaker-service-url", help="Speaker Recognition service URL (default: prompt user)") - parser.add_argument("--parakeet-asr-url", + parser.add_argument("--parakeet-asr-url", help="Parakeet ASR service URL (default: prompt user)") + parser.add_argument("--transcription-provider", + choices=["deepgram", "parakeet", "none"], + help="Transcription provider (default: prompt user)") parser.add_argument("--enable-https", action="store_true", help="Enable HTTPS configuration (default: prompt user)") parser.add_argument("--server-ip", diff --git a/backends/advanced/nginx.conf.template b/backends/advanced/nginx.conf.template deleted file mode 100644 index e5a3e025..00000000 --- a/backends/advanced/nginx.conf.template +++ /dev/null @@ -1,221 +0,0 @@ -worker_processes 1; - -events { - worker_connections 1024; -} - -http { - # Basic settings - sendfile on; - tcp_nopush on; - tcp_nodelay on; - keepalive_timeout 65; - types_hash_max_size 2048; - client_max_body_size 100M; - - # MIME types - include /etc/nginx/mime.types; - default_type application/octet-stream; - - # Logging - access_log /var/log/nginx/access.log; - error_log /var/log/nginx/error.log; - - # Gzip compression - gzip on; - gzip_vary on; - gzip_min_length 10240; - gzip_proxied expired no-cache no-store private auth; - gzip_types - text/plain - text/css - text/xml - text/javascript - application/x-javascript - application/xml+rss - application/javascript - application/json; - - # WebSocket proxy settings - map $http_upgrade $connection_upgrade { - default upgrade; - '' close; - } - - # Upstream services - upstream chronicle_backend { - server chronicle-backend:8000; - } - - upstream friend_webui { - server webui:5173; - } - - # HTTPS Server - server { - listen 443 ssl http2; - server_name localhost TAILSCALE_IP; - - # SSL Configuration - ssl_certificate /etc/nginx/ssl/server.crt; - ssl_certificate_key /etc/nginx/ssl/server.key; - ssl_protocols TLSv1.2 TLSv1.3; - ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384; - ssl_prefer_server_ciphers off; - - # Security headers - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - # Backend API endpoints - location /api/ { - proxy_pass http://chronicle_backend/api/; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_redirect off; - } - - # Authentication endpoints - location /auth/ { - proxy_pass http://chronicle_backend/auth/; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_redirect off; - } - - # Users endpoints - location /users/ { - proxy_pass http://chronicle_backend/users/; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_redirect off; - } - - # WebSocket endpoints for audio streaming - location /ws_pcm { - proxy_pass http://chronicle_backend/ws_pcm; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $connection_upgrade; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_cache_bypass $http_upgrade; - proxy_read_timeout 86400; - proxy_send_timeout 86400; - proxy_connect_timeout 60s; - proxy_buffering off; - proxy_request_buffering off; - } - - location /ws_omi { - proxy_pass http://chronicle_backend/ws_omi; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $connection_upgrade; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_cache_bypass $http_upgrade; - proxy_read_timeout 86400; - proxy_send_timeout 86400; - proxy_connect_timeout 60s; - proxy_buffering off; - proxy_request_buffering off; - } - - # Legacy WebSocket endpoint - location /ws { - proxy_pass http://chronicle_backend/ws; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $connection_upgrade; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_cache_bypass $http_upgrade; - proxy_read_timeout 86400; - proxy_send_timeout 86400; - proxy_connect_timeout 60s; - proxy_buffering off; - proxy_request_buffering off; - } - - # Health check endpoints - location /health { - proxy_pass http://chronicle_backend/health; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - } - - # Readiness check endpoint - location /readiness { - proxy_pass http://chronicle_backend/readiness; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - } - - # Audio file serving - location /audio/ { - proxy_pass http://chronicle_backend/audio/; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_redirect off; - - # Add headers for audio file serving - proxy_set_header Accept-Ranges bytes; - proxy_cache_bypass $http_range; - } - - # Vite HMR WebSocket (specific path) - location /@vite/client { - proxy_pass http://friend_webui/@vite/client; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $connection_upgrade; - proxy_cache_bypass $http_upgrade; - } - - # Frontend Vite dev server (with HMR support) - location / { - proxy_pass http://friend_webui/; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_redirect off; - - # Handle WebSocket upgrade for Vite HMR - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $connection_upgrade; - } - } - - # HTTP redirect to HTTPS - server { - listen 80; - server_name localhost TAILSCALE_IP; - return 301 https://$host$request_uri; - } -} \ No newline at end of file diff --git a/backends/advanced/setup-https.sh b/backends/advanced/setup-https.sh deleted file mode 100755 index b565cddc..00000000 --- a/backends/advanced/setup-https.sh +++ /dev/null @@ -1,336 +0,0 @@ -#!/bin/bash -set -e - -# Chronicle Advanced Backend Initialization Script -# Comprehensive setup for all configuration files and optional services - -# Colors for output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -CYAN='\033[0;36m' -NC='\033[0m' # No Color - -print_info() { - echo -e "${BLUE}[INFO]${NC} $1" -} - -print_success() { - echo -e "${GREEN}[SUCCESS]${NC} $1" -} - -print_warning() { - echo -e "${YELLOW}[WARNING]${NC} $1" -} - -print_error() { - echo -e "${RED}[ERROR]${NC} $1" -} - -print_header() { - echo "" - echo -e "${CYAN}===============================================${NC}" - echo -e "${CYAN}$1${NC}" - echo -e "${CYAN}===============================================${NC}" - echo "" -} - -# Reusable backup helper function -backup_with_timestamp() { - local filepath="$1" - - # Verify the file exists - if [ ! -f "$filepath" ]; then - print_error "Cannot backup '$filepath': file does not exist" - return 1 - fi - - # Generate timestamp (POSIX-safe fallback if needed) - local timestamp - if command -v date >/dev/null 2>&1; then - timestamp=$(date +%Y%m%d_%H%M%S 2>/dev/null) || timestamp=$(date +%Y%m%d_%H%M%S) - else - # POSIX fallback - use current time in seconds since epoch - timestamp="$(date +%s)" - fi - - local backup_path="${filepath}.${timestamp}.backup" - - # Create the backup - if cp "$filepath" "$backup_path"; then - echo "$backup_path" - return 0 - else - print_error "Failed to create backup of '$filepath'" - return 1 - fi -} - -# Check if we're in the right directory -if [ ! -f "pyproject.toml" ] || [ ! -d "src" ]; then - print_error "Please run this script from the backends/advanced directory" - exit 1 -fi - -print_header "Chronicle Advanced Backend Initialization" -echo "This script will help you set up the Chronicle backend with all necessary configurations." -echo "" - -# Function to prompt yes/no -prompt_yes_no() { - local prompt="$1" - local default="$2" - local response - - if [ "$default" = "y" ]; then - prompt="$prompt [Y/n]: " - else - prompt="$prompt [y/N]: " - fi - - read -p "$prompt" response - response=${response:-$default} - - if [[ "$response" =~ ^[Yy]$ ]]; then - return 0 - else - return 1 - fi -} - -# Step 1: Handle .env file -print_header "Step 1: Environment Configuration" -if [ -f ".env" ]; then - print_info ".env file already exists" - if prompt_yes_no "Do you want to update it from template?" "n"; then - backup_path=$(backup_with_timestamp ".env") - if [ $? -eq 0 ]; then - print_info "Backed up existing .env to $backup_path" - cp .env.template .env - print_success ".env created from template" - print_warning "Please edit .env to add your API keys and configuration" - else - print_error "Failed to backup .env file, aborting update" - fi - fi -else - if [ -f ".env.template" ]; then - cp .env.template .env - print_success ".env file created from template" - print_warning "IMPORTANT: Edit .env file to add your API keys:" - echo " - DEEPGRAM_API_KEY (for speech-to-text)" - echo " - OPENAI_API_KEY (for memory extraction)" - echo " - ADMIN_EMAIL and ADMIN_PASSWORD" - echo "" - if prompt_yes_no "Would you like to edit .env now?" "y"; then - ${EDITOR:-nano} .env - fi - else - print_error ".env.template not found!" - exit 1 - fi -fi - -# Step 2: Memory configuration -print_header "Step 2: Memory Configuration" -print_info "Memory settings are managed in config.yml (memory section)." - -# Step 3: Diarization configuration -print_header "Step 3: Diarization Configuration" -if [ -f "diarization_config.json" ]; then - print_info "diarization_config.json already exists" - if prompt_yes_no "Do you want to reset it from template?" "n"; then - backup_path=$(backup_with_timestamp "diarization_config.json") - if [ $? -eq 0 ]; then - print_info "Backed up existing diarization_config.json to $backup_path" - cp diarization_config.json.template diarization_config.json - print_success "diarization_config.json reset from template" - else - print_error "Failed to backup diarization_config.json file, aborting reset" - fi - fi -else - if [ -f "diarization_config.json.template" ]; then - cp diarization_config.json.template diarization_config.json - print_success "diarization_config.json created from template" - else - print_error "diarization_config.json.template not found!" - exit 1 - fi -fi - -# Step 4: HTTPS Setup (optional) -print_header "Step 4: HTTPS Configuration (Optional)" -echo "HTTPS is required for:" -echo " - Microphone access from browsers" -echo " - Remote access via network/Tailscale" -echo " - Secure WebSocket connections" -echo "" - -if prompt_yes_no "Do you want to set up HTTPS?" "n"; then - if [ -f "init-https.sh" ]; then - echo "" - print_info "Please enter your Tailscale IP or network IP" - print_info "Example: 100.83.66.30" - read -p "IP Address: " TAILSCALE_IP - - if [ -n "$TAILSCALE_IP" ]; then - ./init-https.sh "$TAILSCALE_IP" - HTTPS_ENABLED=true - else - print_warning "Skipping HTTPS setup - no IP provided" - HTTPS_ENABLED=false - fi - else - print_warning "init-https.sh not found, skipping HTTPS setup" - HTTPS_ENABLED=false - fi -else - print_info "Skipping HTTPS setup" - HTTPS_ENABLED=false -fi - -# Step 5: Optional Services -print_header "Step 5: Optional Services (extras/)" - -echo "Configure additional services from extras/:" -echo "" - -# Helper function to update or add environment variable in .env file -update_env_var() { - local key=$1 - local value=$2 - - # Use Python to safely update the .env file - python3 -c " -import sys -import re - -key = '$key' -value = '$value' -env_file = '.env' - -# Read existing .env file -try: - with open(env_file, 'r') as f: - lines = f.readlines() -except FileNotFoundError: - lines = [] - -# Check if key exists (uncommented) -updated = False -for i, line in enumerate(lines): - # Skip comments - if line.strip().startswith('#'): - continue - # Check for existing key - if re.match(f'^\\s*{re.escape(key)}=', line): - lines[i] = f'{key}={value}\\n' - updated = True - break - -# If not found, append to end -if not updated: - if lines and not lines[-1].endswith('\\n'): - lines.append('\\n') - lines.append(f'{key}={value}\\n') - -# Write back to file -with open(env_file, 'w') as f: - f.writelines(lines) -" -} - -# OpenMemory MCP (Memory Provider) -if prompt_yes_no "Use OpenMemory MCP for memory management? (cross-client compatible)" "n"; then - update_env_var "MEMORY_PROVIDER" "openmemory_mcp" - print_success "Configured for OpenMemory MCP" - OPENMEMORY_ENABLED=true -else - OPENMEMORY_ENABLED=false -fi - -# Parakeet ASR (Offline Transcription) -if prompt_yes_no "Use Parakeet for offline transcription? (requires GPU)" "n"; then - update_env_var "PARAKEET_ASR_URL" "http://host.docker.internal:8767" - print_success "Configured for Parakeet ASR" - PARAKEET_ENABLED=true -else - PARAKEET_ENABLED=false -fi - -# Speaker Recognition -if prompt_yes_no "Enable Speaker Recognition service?" "n"; then - update_env_var "SPEAKER_SERVICE_URL" "http://host.docker.internal:8001" - print_success "Configured for Speaker Recognition" - SPEAKER_ENABLED=true -else - SPEAKER_ENABLED=false -fi - -# Step 6: Summary and Next Steps -print_header "Setup Complete!" - -echo "Configuration Summary:" -echo "----------------------" -echo "✅ Environment file (.env) configured" -echo "✅ Memory configuration (config.yml) ready" -echo "✅ Diarization configuration (diarization_config.json) ready" - -if [ "$HTTPS_ENABLED" = true ]; then - echo "✅ HTTPS configured with SSL certificates" -fi - -echo "" -echo "Next Steps:" -echo "-----------" - -if [ "$HTTPS_ENABLED" = true ]; then - echo "1. Start the services with HTTPS:" - echo " ${CYAN}docker compose up --build -d${NC}" - echo "" - echo "2. Access the dashboard:" - echo " 🌐 https://localhost/" - echo " 🌐 https://$TAILSCALE_IP/" -else - echo "1. Start the services:" - echo " ${CYAN}docker compose up --build -d${NC}" - echo "" - echo "2. Access the dashboard:" - echo " 🌐 http://localhost:5173" -fi - -echo "" -echo "3. Check service health:" -echo " ${CYAN}curl http://localhost:8000/health${NC}" - -echo "" -if [ "$OPENMEMORY_ENABLED" = true ] || [ "$PARAKEET_ENABLED" = true ] || [ "$SPEAKER_ENABLED" = true ]; then - echo "Start Optional Services:" - echo "------------------------" - - if [ "$OPENMEMORY_ENABLED" = true ]; then - echo "OpenMemory MCP:" - echo " ${CYAN}cd ../../extras/openmemory-mcp && docker compose up -d${NC}" - fi - - if [ "$PARAKEET_ENABLED" = true ]; then - echo "Parakeet ASR:" - echo " ${CYAN}cd ../../extras/asr-services && docker compose up parakeet -d${NC}" - fi - - if [ "$SPEAKER_ENABLED" = true ]; then - echo "Speaker Recognition:" - echo " ${CYAN}cd ../../extras/speaker-recognition && docker compose up --build -d${NC}" - fi -fi - -echo "" -echo "For more information, see:" -echo " - Docs/quickstart.md" -echo " - Docs/memory-configuration-guide.md" -echo " - MEMORY_PROVIDERS.md" - -echo "" -print_success "Initialization complete! 🎉" diff --git a/extras/asr-services/init.py b/extras/asr-services/init.py index 911c527b..d65043cf 100755 --- a/extras/asr-services/init.py +++ b/extras/asr-services/init.py @@ -6,6 +6,7 @@ import argparse import os +import platform import shutil import subprocess import sys @@ -131,10 +132,18 @@ def setup_cuda_version(self): """Configure PyTorch CUDA version""" self.print_section("PyTorch CUDA Version Configuration") + # Detect macOS (Darwin) and auto-default to CPU + is_macos = platform.system() == 'Darwin' + # Check if provided via command line if hasattr(self.args, 'pytorch_cuda_version') and self.args.pytorch_cuda_version: cuda_version = self.args.pytorch_cuda_version self.console.print(f"[green][SUCCESS][/green] PyTorch CUDA version configured from command line: {cuda_version}") + elif is_macos: + # Auto-default to CPU on macOS + cuda_version = "cpu" + self.console.print("[blue][INFO][/blue] Detected macOS - GPU acceleration not available (Apple Silicon/Intel)") + self.console.print("[green][SUCCESS][/green] Using CPU-only PyTorch build") else: # Detect system CUDA version and suggest as default detected_cuda = self.detect_cuda_version() diff --git a/extras/speaker-recognition/init.py b/extras/speaker-recognition/init.py index 8267e35b..b69e04ee 100755 --- a/extras/speaker-recognition/init.py +++ b/extras/speaker-recognition/init.py @@ -7,6 +7,7 @@ import argparse import getpass import os +import platform import shutil import subprocess import sys @@ -188,10 +189,18 @@ def setup_compute_mode(self): """Configure compute mode (CPU/GPU)""" self.print_section("Compute Mode Configuration") + # Detect macOS (Darwin) and auto-default to CPU + is_macos = platform.system() == 'Darwin' + # Check if provided via command line if hasattr(self.args, 'compute_mode') and self.args.compute_mode: compute_mode = self.args.compute_mode self.console.print(f"[green][SUCCESS][/green] Compute mode configured from command line: {compute_mode}") + elif is_macos: + # Auto-default to CPU on macOS + compute_mode = "cpu" + self.console.print("[blue][INFO][/blue] Detected macOS - GPU acceleration not available (Apple Silicon/Intel)") + self.console.print("[green][SUCCESS][/green] Using CPU mode") else: choices = { "1": "CPU-only (works everywhere)", diff --git a/wizard.py b/wizard.py index a2e2b2f7..31a436cd 100755 --- a/wizard.py +++ b/wizard.py @@ -14,7 +14,7 @@ from dotenv import get_key from rich import print as rprint from rich.console import Console -from rich.prompt import Confirm +from rich.prompt import Confirm, Prompt console = Console() @@ -153,18 +153,22 @@ def cleanup_unselected_services(selected_services): console.print(f"🧹 [dim]Backed up {service_name} configuration to {backup_file.name} (service not selected)[/dim]") def run_service_setup(service_name, selected_services, https_enabled=False, server_ip=None, - obsidian_enabled=False, neo4j_password=None): + obsidian_enabled=False, neo4j_password=None, transcription_provider='deepgram'): """Execute individual service setup script""" if service_name == 'advanced': service = SERVICES['backend'][service_name] - + # For advanced backend, pass URLs of other selected services and HTTPS config cmd = service['cmd'].copy() if 'speaker-recognition' in selected_services: cmd.extend(['--speaker-service-url', 'http://speaker-service:8085']) if 'asr-services' in selected_services: cmd.extend(['--parakeet-asr-url', 'http://host.docker.internal:8767']) - + + # Pass transcription provider choice from wizard + if transcription_provider: + cmd.extend(['--transcription-provider', transcription_provider]) + # Add HTTPS configuration if https_enabled and server_ip: cmd.extend(['--enable-https', '--server-ip', server_ip]) @@ -331,6 +335,37 @@ def setup_config_file(): else: console.print("ℹ️ [blue]config/config.yml already exists, keeping existing configuration[/blue]") +def select_transcription_provider(): + """Ask user which transcription provider they want""" + console.print("\n🎤 [bold cyan]Transcription Provider[/bold cyan]") + console.print("Choose your speech-to-text provider:") + console.print() + + choices = { + "1": "Deepgram (cloud-based, high quality, requires API key)", + "2": "Parakeet ASR (offline, runs locally, requires GPU)", + "3": "None (skip transcription setup)" + } + + for key, desc in choices.items(): + console.print(f" {key}) {desc}") + console.print() + + while True: + try: + choice = Prompt.ask("Enter choice", default="1") + if choice in choices: + if choice == "1": + return "deepgram" + elif choice == "2": + return "parakeet" + elif choice == "3": + return "none" + console.print(f"[red]Invalid choice. Please select from {list(choices.keys())}[/red]") + except EOFError: + console.print("Using default: Deepgram") + return "deepgram" + def main(): """Main orchestration logic""" console.print("🎉 [bold green]Welcome to Chronicle![/bold green]\n") @@ -344,9 +379,17 @@ def main(): # Show what's available show_service_status() + # Ask about transcription provider FIRST (determines which services are needed) + transcription_provider = select_transcription_provider() + # Service Selection selected_services = select_services() - + + # Auto-add asr-services if Parakeet was chosen + if transcription_provider == "parakeet" and 'asr-services' not in selected_services: + console.print("[blue][INFO][/blue] Auto-adding ASR services for Parakeet transcription") + selected_services.append('asr-services') + if not selected_services: console.print("\n[yellow]No services selected. Exiting.[/yellow]") return @@ -442,10 +485,10 @@ def main(): success_count = 0 failed_services = [] - + for service in selected_services: if run_service_setup(service, selected_services, https_enabled, server_ip, - obsidian_enabled, neo4j_password): + obsidian_enabled, neo4j_password, transcription_provider): success_count += 1 else: failed_services.append(service)