added websocket and device configuration to this package

[sligara7]

Summary of Changes

This update adds significant new functionality to the bluesky-queueserver-api, focusing on WebSocket communication, device coordination, and device configuration management. Here's a breakdown of the major additions:

1. WebSocket Support

• New WebSocket defaults added to _defaults.py for WebSocket server URI, connection timeouts, heartbeat intervals, and reconnection settings
• Enhanced communication layer in comm_base.py and comm_async.py with new ReManagerAPI_WS_Base class providing WebSocket functionality
• WebSocket API module created at ws/ with both sync (init.py) and async (aio.py) implementations
• Comprehensive WebSocket documentation added to api_docstrings.py with detailed examples for real-time monitoring and device coordination

2. Device Coordination Service (New Package: coordination/)

This is a complete new microservice for managing device access conflicts:

• Service Architecture: FastAPI-based service (service.py) with HTTP and WebSocket APIs
• Client Library: Full-featured client (client.py) with context managers for automatic lock management
• Data Models: Priority-based lock system (models.py) with lock types: MAINTENANCE, QUEUE_EXECUTING, MANUAL_OPERATOR, QUEUE_QUEUED
• Redis Persistence: Lock persistence across service restarts
• Real-time Updates: WebSocket notifications for lock status changes
• Integration Ready: Designed to work with Queue Server and Ophyd WebSocket services

Key Features:

• Priority-based device locking (higher priority locks can override lower ones)
• Automatic timeout and cleanup of expired locks
• Real-time WebSocket notifications for lock changes
• Context managers for safe lock acquisition/release
• Integration with shared device configuration

3. Device Configuration Management

• Device Configuration Manager (device_config.py): Streamlined device discovery using bluesky-queueserver's existing capabilities
• REST API (device_config_api.py): FastAPI endpoints for accessing device configurations
• Multi-format Export: Support for both Queue Server and Ophyd-as-a-Service formats
• Profile Collection Integration: Automatic device discovery from beamline profile collections

4. Enhanced Package Structure

• Optional imports in init.py with graceful fallbacks for missing dependencies
• Modular design allowing components to work independently
• Backward compatibility maintained with existing API structure

5. Integration Points

The changes enable better integration between:

• Queue Server ↔ Ophyd WebSocket Service (via coordination service)
• Queue Server ↔ Device Configuration (shared device definitions)
• Real-time monitoring via WebSocket subscriptions
• Conflict prevention during plan execution and manual device control

6. Deployment and Usage

• Standalone service execution: The coordination service can run independently
• Command-line tools: Scripts for running coordination service
• Docker-ready: Configuration suitable for containerized deployment
• Environment variable support: Configurable via environment variables

This update transforms the bluesky-queueserver-api from a simple HTTP/ZMQ client into a comprehensive ecosystem that supports real-time communication, device coordination, and conflict prevention across multiple services in the Bluesky environment.

Summary: React Queue Monitor

The new react_queue_monitor directory introduces a comprehensive React component library that provides web-based equivalents to the Qt-based Bluesky Queue Server monitoring and control interface. Here's a breakdown of what has been added:

Package Overview

• NPM Package: @bluesky/react-queue-monitor - A complete React library for queue server interaction
• TypeScript Support: Fully typed with comprehensive interface definitions
• WebSocket Integration: Real-time communication with the queue server
• Modular Design: Individual components that can be used independently or together

Core Components

1. Main Dashboard

• QueueMonitorDashboard.tsx: Complete dashboard combining all monitoring and control features
• Supports multiple layouts: monitor, editor, or split view
• Manages UI state for tabs, editing modes, and lock management

2. Individual React Components (Qt equivalents)

• StatusMonitor.tsx: Displays queue server status and connection state
• PlanQueue.tsx: Shows queued plans with editing capabilities (≡ QtRePlanQueue)
• RunningPlan.tsx: Displays currently executing plan details
• ConsoleMonitor.tsx: Real-time console output streaming (≡ QtReConsoleMonitor)
• QueueControls.tsx: Start, stop, pause, resume queue operations
• EnvironmentControls.tsx: Open, close, destroy RE environment
• ExecutionControls.tsx: Stop, abort, halt, pause, resume plan execution
• PlanEditor.tsx: Add and edit plans with parameter validation

3. WebSocket Hook

• useQueueServerSocket.ts: Custom React hook providing:
  • Real-time WebSocket communication
  • State management for queue server data
  • Connection handling with automatic reconnection
  • All queue server API methods wrapped as React functions
  • Request/response handling with timeouts
  • Error handling and connection status tracking

4. Type Definitions

• queueServerTypes.ts: Comprehensive TypeScript interfaces for:
  • Queue items, running plans, history items
  • Manager status, queue status, console output
  • WebSocket messages and API responses
  • Plan and device information structures
  • Lock management types

Key Features

Real-time Communication

• WebSocket-based live updates (queue status, plan execution, console output)
• Automatic reconnection with configurable retry logic
• Request/response pattern with timeout handling

Complete Functionality

• Queue Management: Add, remove, move, edit plans
• Execution Control: Start/stop queue, pause/resume plans, abort/halt operations
• Environment Control: Open/close/destroy RunEngine environments
• Plan Editing: Visual plan editor with parameter validation
• Console Monitoring: Live console output with message filtering
• Status Monitoring: Real-time server and queue status display

Developer Experience

• TypeScript First: Complete type safety and IntelliSense support
• Modular Architecture: Use individual components or complete dashboard
• Configuration Options: Flexible WebSocket and HTTP URL configuration
• Error Handling: Comprehensive error states and user feedback
• Responsive Design: Web-native UI components

Usage Patterns

// Complete Dashboard
<QueueMonitorDashboard wsUrl="ws://localhost:60610" />

// Individual Components
const { status, planQueue, startQueue } = useQueueServerSocket();
<StatusMonitor status={status} />
<PlanQueue planQueue={planQueue} />

// Custom Integration
const hook = useQueueServerSocket({ 
  wsUrl: 'ws://localhost:60610',
  autoConnect: true 
});

Integration Value

This React library enables:

• Web-based Queue Monitoring: Move from Qt desktop to web interfaces
• Dashboard Integration: Embed queue monitoring in larger web applications
• Custom UIs: Build specialized interfaces using individual components
• Real-time Collaboration: Multiple users can monitor queue status simultaneously
• Mobile Support: Queue monitoring on tablets and mobile devices

The React Queue Monitor represents a complete port of the Qt-based queue monitoring functionality to the web, maintaining feature parity while leveraging modern React patterns and WebSocket real-time communication.

[danielballan]

Thanks for this wonderfully detailed PR descrption, @sligara7. There is a lot to unpack in here.

I think this package is a client library, a Python API to QueueServer's REST API. New server code may sit more comfortably in a different repository (the QueueServer repository?)

But I'm missing buckets of context here, so I could be wrong. Looking forward to learning more.

[sligara7]

@danielballan, thanks for the quick comments, particularly on what should stay within the client side (bluesky_queueserver_api) versus moving to the server side (bluesky-queueserver)? Here are some recommendations (with the help of a LLM):

✅ KEEP in bluesky-queueserver-api:

1. device_config.py ✅
   Shared utility that can be used by both client and server contexts
   Lightweight dependencies, no server frameworks
2. client.py ✅
   Client library for connecting TO coordination services
   Used by external services like ophyd-websocket
   Perfect fit for client package
3. models.py ✅
   Shared data models (DeviceLock, LockRequest, etc.)
   Used by both client and server
   Common pattern to keep shared models in client package
   Only depends on Pydantic (lightweight)

❌ MOVE to bluesky-queueserver:
4. device_config_api.py ❌
This is SERVER code - provides FastAPI REST API endpoints
Creates global server state (_device_manager)
Dependencies: FastAPI, Pydantic (server frameworks)
Already imported by coordination service to mount routes
5. service.py ❌
Server implementation running FastAPI app
Dependencies: FastAPI, uvicorn, aioredis (heavyweight server deps)
Should be deployed alongside queue server
6. run_coordination_service.py ❌
Server startup script
Follows service.py to server package
🎯 Key Insight on device_config_api.py:
This file creates FastAPI routes and manages global server state:

router = APIRouter(prefix="/api/device-config", tags=["device-config"])
_device_manager: Optional[DeviceConfigurationManager] = None

Even though it provides "API" functionality, it's server-side API code (FastAPI routers), not client API code. The coordination service already imports and mounts these routes, confirming it's a server component.

📋 Final Structure:
Stay in bluesky-queueserver-api:
device_config.py - Shared utility
client.py - Client library
models.py - Shared models

Move to bluesky-queueserver:
device_config_api.py → device_config_api.py
service.py → servers/coordination_service.py
run_coordination_service.py → servers/run_coordination.py
This maintains clean client/server separation while keeping the shared components (models, client, utility) accessible to both packages.

Ultimately trying to have an interface between: Ophyd as a Service (https://github.com/bluesky/ophyd-websocket/tree/feature/oas) to create something like the following:

This is another drawing:

Lots of architectural decisions to make about how to build this out, so looking forward to any recommendations you or anyone else may have.

[sligara7]

Reanalyzing the architecture, I think many features belong in different packages within the bluesky-ecosystem. Thanks for the comments.