Epic: Cross-platform daemon/service mode for daemoneye-agent (Unix/macOS/Windows) with process supervision

[unclesp1d3r]

Cross-platform Daemon/Service Mode for daemoneye-agent

Summary

Implement a first-class, cross-platform service/daemon mode for the daemoneye-agent (Orchestrator) using the proc-daemon crate that:

• Runs as a native background service on Linux/macOS (daemon) and Windows (service)
• Supervises collector processes (e.g., procmond and specialty collectors) with robust lifecycle management
• Integrates with system service managers (systemd, launchd, Windows SCM)
• Preserves parity with interactive mode while adding secure privilege management, logging, health, and recovery

🏗️ Architecture Integration: This issue implements the service foundation that enables daemoneye-agent to manage collector-core based components like procmond as described in issue #89.

Context & Motivation

This is a core building block to operate DaemonEye reliably on servers and endpoints, ensuring automatic startup at boot, robust recovery, and centralized control of all monitoring components.

• Critical for production deployments and unattended operation (blocking adoption)
• Ensures proper privilege separation and least-privilege runtime
• Enables standardized installation, logging, and observability across Linux, macOS, and Windows
• Provides resilient supervision for collector processes with health checks and backoff restarts

Architecture Integration

Service Hierarchy

Operating System
├── Service Manager (systemd/launchd/SCM)
│   └── daemoneye-agent (proc-daemon based service)
│       ├── Process Supervision
│       │   ├── procmond (collector-core based)
│       │   ├── netmond (collector-core based)  
│       │   └── filemond (collector-core based)
│       ├── Configuration Management
│       ├── Health Monitoring
│       └── IPC Coordination
└── System APIs & Resources

Integration Points

• Issue Implement Core Process Monitoring Daemon (procmond) #89: procmond implementation will run as managed collector under this service
• collector-core: All collectors use this framework and are supervised by daemoneye-agent
• proc-daemon: Core service behavior and cross-platform daemon functionality
• daemoneye-lib: Shared configuration, IPC, and telemetry infrastructure

Technical Requirements

Service Framework Implementation

• proc-daemon Integration
  • Integrate proc-daemon crate for cross-platform service behavior
  • Handle service installation, start, stop, restart operations
  • Implement service status reporting and health monitoring
  • Support service configuration management

Cross-Platform Service Management

• Linux/macOS Daemon Support

  • systemd service integration with proper service files
  • launchd integration for macOS
  • UNIX signal handling (SIGTERM, SIGHUP, SIGUSR1)
  • PID file management and process locking

• Windows Service Support

  • Windows Service Control Manager (SCM) integration
  • Service installation and uninstallation
  • Windows service events and logging
  • Service recovery configuration

Collector Process Supervision

• Lifecycle Management

  • Start, stop, restart collector processes (procmond, etc.)
  • Health monitoring and automated recovery
  • Graceful shutdown coordination
  • Process resource monitoring and limits

• Configuration Management

  • Hot-reload configuration without service restart
  • Per-collector configuration validation
  • Configuration change propagation
  • Environment isolation for collectors

Implementation Phases

Phase 1: Service Foundation

1. proc-daemon Integration

   • Add proc-daemon crate dependency to daemoneye-agent
   • Implement basic service lifecycle (start, stop, status)
   • Create service installation/uninstallation logic
   • Add cross-platform service configuration

2. Platform Integration

   • Create systemd service files for Linux
   • Implement launchd plist for macOS
   • Add Windows service manifest and installer
   • Test service registration on all platforms

Phase 2: Process Supervision

1. Collector Management

   • Implement collector process spawning and monitoring
   • Add health check integration with collector-core
   • Create configuration distribution system
   • Implement graceful shutdown coordination

2. Recovery & Resilience

   • Add automatic restart capabilities
   • Implement circuit breaker patterns
   • Create resource monitoring and cleanup
   • Add performance metrics collection

Dependencies

• proc-daemon: Core service/daemon functionality
• collector-core: Framework for all managed collectors
• daemoneye-lib: Shared infrastructure (config, IPC, telemetry)
• procmond: Primary collector to be supervised (Issue Implement Core Process Monitoring Daemon (procmond) #89)
• Platform service managers (systemd, launchd, Windows SCM)

Acceptance Criteria

Service Operation

• Cross-platform Compatibility

  • Successfully installs and runs on Linux, macOS, and Windows
  • Integrates with native service managers on each platform
  • Handles platform-specific service requirements

• Service Lifecycle

  • Service starts automatically at boot
  • Responds correctly to start, stop, restart commands
  • Handles graceful shutdown and cleanup
  • Supports service status queries

Collector Supervision

• Process Management
  • Successfully starts and monitors collector processes
  • Detects and recovers from collector failures
  • Coordinates graceful shutdown of all components
  • Reports aggregate health status

Timeline

Target completion aligned with v0.2.0 milestone (Due: September 22, 2025)

Related Issues

• Issue Implement Core Process Monitoring Daemon (procmond) #89: Procmond implementation (primary supervised collector)
• Issue Implement Comprehensive CLI Interface and Management System #91: CLI interface integration with service mode
• Issue 🧪 Implement Comprehensive Test Suite with Quality Assurance Pipeline #61: Testing framework for service reliability
• Issue Implement Tamper-Evident Audit Logging System with BLAKE3 Cryptographic Chain #42: Audit logging integration

---

This issue enables the core service architecture that makes DaemonEye a production-ready monitoring platform.

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

---

I'll help enhance this issue with additional context, improve the title, and add appropriate metadata. Let me first check the current state and available labels.

---

🏁 Script executed:

#!/bin/bash
# First, let's check the current state of the issue and available labels
gh issue view 103 --json title,body,labels,assignees,milestone,state

# Also check what labels are available in the repository
gh label list --limit 100

Length of output: 9333

---

@unclesp1d3r Applying the enhanced title, enriched body, labels, and assignee now. I’ll report back with a summary once applied.

---

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

issue_number=103
new_title="Epic: Cross-platform daemon/service mode for daemoneye-agent (Unix/macOS/Windows) with process supervision"

tmp_body="$(mktemp)"
cat > "$tmp_body" << 'EOF'
## Summary

Implement a first-class, cross-platform service/daemon mode for the daemoneye-agent (Orchestrator) that:
- Runs as a native background service on Linux/macOS (daemon) and Windows (service)
- Supervises collector processes (e.g., procmond and specialty collectors) with robust lifecycle management
- Integrates with system service managers (systemd, launchd, Windows SCM)
- Preserves parity with interactive mode while adding secure privilege management, logging, health, and recovery

This is a core building block to operate DaemonEye reliably on servers and endpoints, ensuring automatic startup at boot, robust recovery, and centralized control.

## Motivation / Context

- Critical for production deployments and unattended operation (blocking adoption).
- Ensures proper privilege separation and least-privilege runtime.
- Enables standardized installation, logging, and observability across Linux, macOS, and Windows.
- Provides resilient supervision for collector processes with health checks and backoff restarts.
- Aligns with requirements 6.1, 6.2, 6.4, 6.5, 10.x, 11.x, 12.x.

## Goals

- Native daemon/service integration on Unix-like systems and Windows.
- Privilege dropping, PID file management (Unix), SCM registration (Windows), and signal/control handling.
- Process supervision for collectors with graceful shutdown, crash detection, and backoff restart.
- Health, status, and metrics reporting suitable for external monitoring.
- CLI-driven service lifecycle operations and configuration validation.
- Parity with interactive mode for functional behavior and IPC.

## Non-Goals

- Packaging/distribution (e.g., MSI/DEB/RPM) beyond minimal service templates.
- Cluster/orchestrator HA design (single-host scope only).
- Self-update/upgrader beyond standard deployment instructions.

## High-level Design

```mermaid
flowchart LR
  subgraph OS
    Systemd-.->SVC
    Launchd-.->SVC
    SCM[Windows SCM]-.->SVC
  end
  SVC[daemoneye-agent Service/Daemon]
  SVC --> SUP[ProcessSupervisor]
  SUP -->|spawn/monitor| COLS[Collectors (procmond, specialty)]
  SVC -->|IPC| DET[Detection Engine]
  SVC -->|health/metrics| MON[External Monitoring]
  SVC --> LOG[(Rotating Log Files)]
```

- ServiceManager trait abstracts platform-specific lifecycle (UnixServiceManager, WindowsServiceManager).
- ProcessSupervisor manages child collector processes, enforces policies, and aggregates status.
- Configuration-driven startup (working directory, user context, env), with validation and diagnostics at boot.

## Scope & Deliverables

- [ ] 21.1 Add daemon/service dependencies and cross-platform service infrastructure
  - Add `daemonize` (Unix) and `windows-service` (Windows)
  - `ServiceManager` trait; `UnixServiceManager`, `WindowsServiceManager`
  - Service config structure (startup options, working directory, user context)
  - Unit tests for trait impls + config validation
  - Requirements: 6.1, 6.2, 6.4, 6.5

- [ ] 21.2 Implement Unix daemon functionality with proper privilege management
  - Daemon init: fork, setsid, chdir, umask, FD mgmt
  - PID file w/ advisory lock and cleanup
  - Signals: SIGTERM, SIGINT, SIGHUP (reload), SIGUSR1 (status)
  - Drop privileges post-init with audit logging
  - Graceful shutdown of collectors
  - Integration tests for lifecycle + signals
  - Requirements: 6.1, 6.2, 6.4, 6.5

- [ ] 21.3 Implement Windows service functionality with SCM integration
  - `WindowsService` using windows-service crate
  - SCM registration, control handlers (start/stop/pause/continue/shutdown)
  - Windows Event Log integration
  - Installer/uninstaller helpers with registry entries
  - Recovery options and failure policies
  - Integration tests for lifecycle + SCM interaction
  - Requirements: 6.1, 6.2, 6.4, 6.5

- [ ] 21.4 Add collector process lifecycle management and supervision
  - `ProcessSupervisor` for procmond and specialty collectors
  - Spawn with controlled env and privileges
  - Health monitoring (heartbeat), restart policies with exponential backoff
  - Coordinated graceful shutdown with timeouts
  - Crash detection and auto-restart
  - Resource monitoring and limits
  - Integration tests for recovery scenarios
  - Requirements: 11.1, 11.2, 12.1, 12.2

- [ ] 21.5 Integrate service functionality with existing daemoneye-agent architecture
  - main.rs supports interactive and service modes
  - CLI flags: `--install`, `--uninstall`, `--start`, `--stop`, `--status`
  - Integrate with config system and DB init; preserve IPC and detection engine
  - Service-specific logging with file rotation and retention
  - Templates for systemd, launchd, and Windows services
  - Integration tests for parity with interactive mode
  - Requirements: 6.1, 6.2, 6.4, 6.5, 11.1, 11.2

- [ ] 21.6 Add service monitoring and health reporting capabilities
  - Health endpoints for external systems
  - Service status reporting with component aggregation
  - Metrics export for collector status and performance
  - Config validation and startup diagnostics
  - Log aggregation across managed collectors
  - Recovery actions for common failures
  - Monitoring integration tests (simulated failures + validation)
  - Requirements: 10.1–10.4

- [ ] 21.7 Create service deployment and configuration management
  - systemd unit files (Linux) with deps and security hardening
  - launchd plists (macOS) with proper permissions
  - Windows service installer with registry + security descriptors
  - Deployment validation and configuration checks
  - Update/migration procedures for config changes
  - Deployment documentation per platform
  - Automation scripts for common scenarios
  - Requirements: 6.1, 6.2, 6.4, 6.5

- [ ] 21.8 Add comprehensive service testing and validation
  - Install/start/stop tests across platforms
  - Failure scenario tests (collector crashes, recovery)
  - Upgrade tests with config migration and data preservation
  - Security tests for privilege boundaries and access controls
  - Performance tests under high load with multiple collectors
  - Compatibility tests across OS versions
  - Chaos tests for failure modes and recovery
  - Requirements: all requirements verification

## Acceptance Criteria

- Unix:
  - Daemon starts via systemd/launchd and logs to rotating files; PID file created/cleaned; signals handled (TERM/INT: graceful stop, HUP: reload).
- Windows:
  - Service installs/uninstalls; responds to SCM controls (start/stop/pause/continue/shutdown); events logged to Windows Event Log.
- Cross-platform:
  - CLI service ops available: `--install`, `--uninstall`, `--start`, `--stop`, `--status`.
  - ProcessSupervisor monitors at least one collector, detects crashes, restarts with backoff, and coordinates graceful shutdowns within timeouts.
  - Health endpoint returns aggregated component status; metrics expose collector lifecycle/health.
  - Service mode functionally matches interactive mode for IPC, detection engine behavior, and configuration semantics.
  - Configuration validation prevents unsafe/invalid startup and emits actionable diagnostics.

## Risks and Mitigations

- Privilege management: enforce drop to dedicated user, validate file perms/ownership, and audit privilege transitions.
- PID file races/stale locks (Unix): use advisory locks and cleanup on exit; detect stale PID files on startup.
- FD/console handling on daemonization: explicitly close/inherit FDs as needed; ensure logging sinks initialized early.
- Windows SCM complexity: robust control handler state machine and time-bounded transitions with retries.
- Restart storms: exponential backoff, jitter, and max-retry caps per collector.
- Log growth: implement rotation + retention policies; ensure write permissions post-privilege drop.

## Dependencies

- Crates: `daemonize` (Unix), `windows-service` (Windows).
- OS integrations: systemd (Linux), launchd (macOS), Windows SCM/Event Log.
- Existing subsystems: configuration, IPC, detection engine, database init.

## CLI

- `daemoneye-agent --install|--uninstall|--start|--stop|--status`
- `--service-config <path>` (optional)
- `--foreground` to run in interactive mode for debugging

## Templates (examples)

- systemd unit (excerpt):
  ```
  [Unit]
  Description=DaemonEye Agent
  After=network.target

  [Service]
  User=daemoneye
  Group=daemoneye
  ExecStart=/usr/bin/daemoneye-agent --service-config /etc/daemoneye/agent.yaml
  Restart=on-failure
  RestartSec=5s
  CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_SYS_PTRACE
  ProtectHome=yes
  ProtectSystem=full
  NoNewPrivileges=true

  [Install]
  WantedBy=multi-user.target
  ```

## Testing Strategy

- Unit tests for ServiceManager, config validation, and supervisor policies.
- Integration tests per OS for install/start/stop, signals/SCM controls, and parity checks.
- Fault-injection tests for collector crashes, timeouts, and backoff restart.
- Performance/soak tests with multiple collectors.

## Release / Migration

- Provide platform templates and scripts for service install/uninstall.
- Document upgrade paths (config migrations, where applicable).
- Validate startup against current configuration and existing data directories.

## Contribution

- @-mentions welcome for code review and testing on target platforms.
- This issue is an Epic; sub-issues/PRs highly encouraged and can be linked here.

EOF

# Apply labels and assignee
labels_to_add=(
  "epic"
  "core-feature"
  "cross-platform"
  "process-monitoring"
  "configuration"
  "integration"
  "testing"
  "priority:high"
)

# Build repeated --add-label flags
label_flags=()
for l in "${labels_to_add[@]}"; do
  label_flags+=(--add-label "$l")
done

# Edit the issue
gh issue edit "$issue_number" \
  --title "$new_title" \
  --body-file "$tmp_body" \
  "${label_flags[@]}" \
  --add-assignee "unclesp1d3r"

# Show a concise summary of changes
gh issue view "$issue_number" --json title,assignees,labels --jq '
{
  title: .title,
  assignees: [.assignees[].login],
  labels: [.labels[].name]
}
'

Length of output: 340

---

@unclesp1d3r Perfect! I've successfully enhanced issue #103 with significant improvements:

What Changed

Title: Upgraded from a generic feature request to:

"Epic: Cross-platform daemon/service mode for daemoneye-agent (Unix/macOS/Windows) with process supervision"

Enhanced Body: Added comprehensive sections including:

• Summary & Context: Clear motivation for why this is blocking adoption
• Goals & Non-Goals: Explicit scope boundaries
• Architecture Diagram: Visual Mermaid flowchart showing service relationships
• Detailed Acceptance Criteria: Platform-specific requirements (Unix signals, Windows SCM, etc.)
• Risk Assessment: Security, privilege management, restart storms, and mitigations
• CLI Interface: Specific command examples (--install, --uninstall, etc.)
• Service Templates: Real systemd unit file example with security hardening
• Testing Strategy: Unit, integration, fault-injection, and performance tests

Labels Applied:

• epic - This is clearly a multi-phase, cross-cutting effort
• core-feature - Essential system functionality
• cross-platform - Unix/macOS/Windows support
• process-monitoring - Collector supervision capabilities
• configuration - Service config management
• integration - System service manager integration
• testing - Comprehensive test strategy required
• priority:high - Critical for adoption (as noted in original)

Assignment: Assigned to you since you volunteered to help implement, test, and document this feature.

The original detailed scope (tasks 21.1-21.8) has been preserved under "Scope & Deliverables" while adding the strategic context needed for an epic of this magnitude. This should provide much better clarity for contributors and stakeholders! 🐰