Skip to content

Fix deepgram/parakeet workers bug #199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AnkushMalaker merged 5 commits into from
Dec 18, 2025
Merged

Fix deepgram/parakeet workers bug #199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ddec370
Enhance worker management in start-workers.sh and update docker-compo…
AnkushMalaker Dec 18, 2025
d25c341
Enhance audio stream worker management in start-workers.sh
AnkushMalaker Dec 18, 2025
ef98703
Update Caddyfile.template to replace friend-backend with chronicle-ba…
AnkushMalaker Dec 18, 2025
6bb8ff8
Update docker-compose.yml files to use chronicle-network
AnkushMalaker Dec 18, 2025
5e2220b
Update docker-compose.yml to use chronicle-network for speaker recogn…
AnkushMalaker Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions backends/advanced/Caddyfile.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ localhost TAILSCALE_IP {

# WebSocket endpoints - proxy to backend with upgrade support
handle /ws* {
reverse_proxy friend-backend:8000 {
reverse_proxy chronicle-backend:8000 {
# Caddy automatically handles WebSocket upgrades
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
Expand All @@ -29,31 +29,31 @@ localhost TAILSCALE_IP {

# API endpoints - proxy to backend
handle /api/* {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

# Auth endpoints - proxy to backend
handle /auth/* {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

# Health checks - proxy to backend
handle /health {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

handle /readiness {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

# Users endpoints - proxy to backend
handle /users/* {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

# Audio files - proxy to backend
handle /audio/* {
reverse_proxy friend-backend:8000
reverse_proxy chronicle-backend:8000
}

# Everything else - proxy to webui
Expand All @@ -68,36 +68,36 @@ localhost TAILSCALE_IP {
#
# # WebSocket endpoints
# handle /ws* {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # API endpoints
# handle /api/* {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # Auth endpoints
# handle /auth/* {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # Health checks
# handle /health {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# handle /readiness {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # Users endpoints
# handle /users/* {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # Audio files
# handle /audio/* {
# reverse_proxy friend-backend:8000
# reverse_proxy chronicle-backend:8000
# }
#
# # Everything else - webui
Expand Down
2 changes: 1 addition & 1 deletion backends/advanced/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ services:
# Shared network for cross-project communication
networks:
default:
name: friend-network
name: chronicle-network

volumes:
ollama_data:
Expand Down
36 changes: 29 additions & 7 deletions backends/advanced/start-workers.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,26 @@ start_workers() {
uv run python -m advanced_omi_backend.workers.rq_worker_entry audio &
AUDIO_PERSISTENCE_WORKER_PID=$!

echo "🎵 Starting audio stream Deepgram worker (1 worker for sequential processing)..."
uv run python -m advanced_omi_backend.workers.audio_stream_deepgram_worker &
AUDIO_STREAM_WORKER_PID=$!
# Start stream workers based on available configuration
# Only start Deepgram worker if DEEPGRAM_API_KEY is set
if [ -n "$DEEPGRAM_API_KEY" ]; then
echo "🎵 Starting audio stream Deepgram worker (1 worker for sequential processing)..."
uv run python -m advanced_omi_backend.workers.audio_stream_deepgram_worker &
AUDIO_STREAM_DEEPGRAM_WORKER_PID=$!
else
echo "⏭️ Skipping Deepgram stream worker (DEEPGRAM_API_KEY not set)"
AUDIO_STREAM_DEEPGRAM_WORKER_PID=""
fi

# Only start Parakeet worker if PARAKEET_ASR_URL or OFFLINE_ASR_TCP_URI is set
if [ -n "$PARAKEET_ASR_URL" ] || [ -n "$OFFLINE_ASR_TCP_URI" ]; then
echo "🎵 Starting audio stream Parakeet worker (1 worker for sequential processing)..."
uv run python -m advanced_omi_backend.workers.audio_stream_parakeet_worker &
AUDIO_STREAM_PARAKEET_WORKER_PID=$!
else
echo "⏭️ Skipping Parakeet stream worker (PARAKEET_ASR_URL/OFFLINE_ASR_TCP_URI not set)"
AUDIO_STREAM_PARAKEET_WORKER_PID=""
fi

echo "✅ All workers started:"
echo " - RQ worker 1: PID $RQ_WORKER_1_PID (transcription, memory, default)"
Expand All @@ -63,7 +80,8 @@ start_workers() {
echo " - RQ worker 5: PID $RQ_WORKER_5_PID (transcription, memory, default)"
echo " - RQ worker 6: PID $RQ_WORKER_6_PID (transcription, memory, default)"
echo " - Audio persistence worker: PID $AUDIO_PERSISTENCE_WORKER_PID (audio queue - file rotation)"
echo " - Audio stream worker: PID $AUDIO_STREAM_WORKER_PID (Redis Streams consumer - sequential processing)"
[ -n "$AUDIO_STREAM_DEEPGRAM_WORKER_PID" ] && echo " - Deepgram stream worker: PID $AUDIO_STREAM_DEEPGRAM_WORKER_PID (real-time transcription)"
[ -n "$AUDIO_STREAM_PARAKEET_WORKER_PID" ] && echo " - Parakeet stream worker: PID $AUDIO_STREAM_PARAKEET_WORKER_PID (real-time transcription)"
}

# Function to check worker registration health
Expand Down Expand Up @@ -103,7 +121,9 @@ monitor_worker_health() {
echo "🔧 Self-healing: Restarting all workers to restore registration..."

# Kill all workers
kill $RQ_WORKER_1_PID $RQ_WORKER_2_PID $RQ_WORKER_3_PID $RQ_WORKER_4_PID $RQ_WORKER_5_PID $RQ_WORKER_6_PID $AUDIO_PERSISTENCE_WORKER_PID $AUDIO_STREAM_WORKER_PID 2>/dev/null || true
kill $RQ_WORKER_1_PID $RQ_WORKER_2_PID $RQ_WORKER_3_PID $RQ_WORKER_4_PID $RQ_WORKER_5_PID $RQ_WORKER_6_PID $AUDIO_PERSISTENCE_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_DEEPGRAM_WORKER_PID" ] && kill $AUDIO_STREAM_DEEPGRAM_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_PARAKEET_WORKER_PID" ] && kill $AUDIO_STREAM_PARAKEET_WORKER_PID 2>/dev/null || true
wait 2>/dev/null || true

# Restart workers
Expand All @@ -128,7 +148,8 @@ shutdown() {
kill $RQ_WORKER_5_PID 2>/dev/null || true
kill $RQ_WORKER_6_PID 2>/dev/null || true
kill $AUDIO_PERSISTENCE_WORKER_PID 2>/dev/null || true
kill $AUDIO_STREAM_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_DEEPGRAM_WORKER_PID" ] && kill $AUDIO_STREAM_DEEPGRAM_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_PARAKEET_WORKER_PID" ] && kill $AUDIO_STREAM_PARAKEET_WORKER_PID 2>/dev/null || true
wait
echo "✅ All workers stopped"
exit 0
Expand Down Expand Up @@ -161,7 +182,8 @@ kill $RQ_WORKER_4_PID 2>/dev/null || true
kill $RQ_WORKER_5_PID 2>/dev/null || true
kill $RQ_WORKER_6_PID 2>/dev/null || true
kill $AUDIO_PERSISTENCE_WORKER_PID 2>/dev/null || true
kill $AUDIO_STREAM_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_DEEPGRAM_WORKER_PID" ] && kill $AUDIO_STREAM_DEEPGRAM_WORKER_PID 2>/dev/null || true
[ -n "$AUDIO_STREAM_PARAKEET_WORKER_PID" ] && kill $AUDIO_STREAM_PARAKEET_WORKER_PID 2>/dev/null || true
wait

echo "🔄 All workers stopped"
Expand Down
20 changes: 13 additions & 7 deletions extras/openmemory-mcp/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,9 @@ services:
- ./data/mem0_storage:/qdrant/storage
restart: unless-stopped

# OpenMemory MCP Server (built from local cache)
# OpenMemory MCP Server (official Docker image)
openmemory-mcp:
build:
context: ./cache/mem0/openmemory/api
dockerfile: Dockerfile
image: mem0/openmemory-mcp:latest
env_file:
- .env
environment:
Expand All @@ -29,8 +27,11 @@ services:
timeout: 10s
retries: 3
start_period: 30s
networks:
- default
- chronicle-network

# OpenMemory UI (optional - can be disabled if not needed)
# OpenMemory UI (starts by default with the MCP server)
openmemory-ui:
image: mem0/openmemory-ui:latest
ports:
Expand All @@ -40,5 +41,10 @@ services:
- NEXT_PUBLIC_USER_ID=openmemory
depends_on:
- openmemory-mcp
profiles:
- ui # Only starts when --profile ui is used
restart: unless-stopped

networks:
default:
name: openmemory-mcp_default
chronicle-network:
external: true
Comment on lines +46 to +50
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

The external chronicle-network dependency requires backend services to start first.

The chronicle-network is marked external: true, which means this compose file depends on the network being pre-created by backend services. This matches the pattern in extras/speaker-recognition/docker-compose.yml and creates an implicit service startup order requirement. Users must start the backend services before running docker compose up in this directory.

🤖 Prompt for AI Agents 
In extras/openmemory-mcp/docker-compose.yml around lines 46 to 50, the
chronicle-network is marked external: true which creates an implicit dependency
on backend services having pre-created that network; change the network to be
created by this compose file instead of requiring an external network by
removing external: true and defining the network locally (set an explicit name
and optional driver) so docker compose up will create it automatically; ensure
the network section uses a local definition consistent with other compose files
(e.g., provide name: openmemory-chronicle-network and remove external: true).

31 changes: 0 additions & 31 deletions extras/openmemory-mcp/init-cache.sh
View file
Open in desktop

This file was deleted.

25 changes: 2 additions & 23 deletions extras/openmemory-mcp/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,24 +43,6 @@ fi
# Set restrictive permissions (owner read/write only)
chmod 600 .env

# Clone the custom fork of mem0 with OpenMemory fixes
echo ""
echo "📦 Setting up custom mem0 fork with OpenMemory..."
if [ -d "cache/mem0" ]; then
echo " Removing existing mem0 directory..."
rm -rf cache/mem0
fi

echo " Cloning mem0 fork from AnkushMalaker/mem0..."
mkdir -p cache
git clone https://github.com/AnkushMalaker/mem0.git cache/mem0
cd cache/mem0
echo " Checking out fix/get-endpoint branch..."
git checkout fix/get-endpoint
cd ../..

echo "✅ Custom mem0 fork ready with OpenMemory improvements"

# Get OpenAI API Key (prompt only if not provided via command line)
if [ -z "$OPENAI_API_KEY" ]; then
echo ""
Expand Down Expand Up @@ -91,9 +73,6 @@ echo ""
echo "✅ OpenMemory MCP configured!"
echo "📁 Configuration saved to .env"
echo ""
echo "🚀 To start: docker compose up --build -d"
echo "🚀 To start: docker compose up -d"
echo "🌐 MCP Server: http://localhost:8765"
echo "📱 Web Interface: http://localhost:8765"
echo "🔧 UI (optional): docker compose --profile ui up -d"
echo ""
echo "💡 Note: Using custom mem0 fork from AnkushMalaker/mem0:fix/get-endpoint"
echo "📱 Web UI: http://localhost:3001"
2 changes: 1 addition & 1 deletion extras/speaker-recognition/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,5 +108,5 @@ services:
# Shared network for cross-project communication
networks:
default:
name: friend-network
name: chronicle-network
external: true
Comment on lines +111 to 112
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for docker-compose files and look for chronicle-network
echo "=== Finding docker-compose files ==="
find . -name "docker-compose*.yml" -o -name "docker-compose*.yaml" | head -20

echo -e "\n=== Searching for chronicle-network in compose files ==="
rg -l "chronicle-network" -t yaml

echo -e "\n=== Examining network definitions ==="
rg "networks:" -A 5 -t yaml | head -50

echo -e "\n=== Looking for documentation about startup order ==="
find . -name "*.md" -o -name "*.txt" -o -name "README*" | xargs grep -l "startup\|start.*backend\|chronicle-network" 2>/dev/null | head -10

Repository: chronicler-ai/chronicle

Length of output: 3238

🏁 Script executed:

#!/bin/bash
# Check documentation files for startup order guidance
echo "=== Checking getting-started.md ==="
grep -i "startup\|start.*backend\|docker.*compose\|order\|first\|second" ./Docs/getting-started.md | head -15

echo -e "\n=== Checking quickstart.md ==="
grep -i "startup\|start.*backend\|docker.*compose\|order\|first\|second" ./backends/advanced/Docs/quickstart.md | head -15

echo -e "\n=== Checking README.md ==="
grep -i "startup\|start.*backend\|docker.*compose\|order\|first\|second" ./README.md | head -15

echo -e "\n=== Full content of getting-started.md (first 100 lines) ==="
head -100 ./Docs/getting-started.md

echo -e "\n=== Checking if speaker-recognition or extras are mentioned in getting-started ==="
grep -i "speaker\|extras\|asr\|recognition" ./Docs/getting-started.md

Repository: chronicler-ai/chronicle

Length of output: 5977

🏁 Script executed:

#!/bin/bash
# Check the specific lines 111-112 in speaker-recognition/docker-compose.yml
echo "=== Lines 111-112 of extras/speaker-recognition/docker-compose.yml ==="
sed -n '110,115p' ./extras/speaker-recognition/docker-compose.yml

echo -e "\n=== Context around network definition (lines 105-115) ==="
sed -n '105,115p' ./extras/speaker-recognition/docker-compose.yml

echo -e "\n=== Network definition in backends/advanced/docker-compose.yml ==="
grep -A 3 "^networks:" ./backends/advanced/docker-compose.yml

echo -e "\n=== Verify backends/advanced network does NOT have external: true ==="
grep -A 3 "^networks:" ./backends/advanced/docker-compose.yml | grep -i external || echo "No 'external' keyword found - network is created by this compose file"

Repository: chronicler-ai/chronicle

Length of output: 665

Document the chronicle-network external dependency requirement.

The chronicle-network is marked as external: true, requiring it to exist before this compose stack starts. While getting-started.md shows the correct startup sequence (backends/advanced first, then extras), add explicit documentation explaining that speaker-recognition depends on the network created by backends/advanced. This prevents confusion if users attempt to run speaker-recognition independently.

🤖 Prompt for AI Agents 
In extras/speaker-recognition/docker-compose.yml around lines 111 to 112, the
compose file marks the chronicle-network as external:true but doesn't document
this dependency; update the repository docs by adding a brief note in
extras/speaker-recognition/README.md (or getting-started.md near the
speaker-recognition section) stating that speaker-recognition requires the
chronicle-network to be created by running backends/advanced first (include the
exact docker network name chronicle-network and the startup order:
backends/advanced -> extras/speaker-recognition), and add a one-line reminder in
the docker-compose.yml header comment referencing that external network and
linking to the doc for the startup steps.