Skip to content

Docker backup script for NAS systems with parallelization, error handling, and automatic container management. Uses rsync for data synchronization, supports ACL preservation, and includes fallback mechanisms for broad compatibility.

License

Notifications You must be signed in to change notification settings

florian101010/NAS-Docker-Backup-rsync

Repository files navigation

🐳 NAS - Docker Backup Script (rsync)

🌍 Choose Your Language / Sprache wÀhlen

English Deutsch


English Version

Version License Bash Platform Downloads Stars

Automated Docker backup script for NAS systems - gracefully stops containers, synchronizes data with rsync, and restarts everything.

🎯 Perfect for: Home labs β€’ Small businesses β€’ Production environments β€’ Any Docker setup on NAS devices

πŸ† Why choose this script: Traditional backup methods corrupt Docker data when containers are running. This script solves that problem by intelligently managing your entire Docker ecosystem - automatically discovering containers, gracefully stopping them for data consistency, creating comprehensive backups of everything (stacks, volumes, configs), and seamlessly restarting services.


πŸš€ Key Features

🐳 Smart Docker Management

  • πŸ” Automatic Container Discovery: Finds all Docker Compose stacks and containers automatically
  • ⏸️ Graceful Container Shutdown: Safely stops containers to prevent data corruption during backup
  • πŸ”„ Intelligent Restart: Automatically restarts all services after backup completion
  • πŸ“¦ Complete Stack Backup: Backs up Docker Compose files, volumes, and persistent data (networks recreated by Compose when using down; with --use-stop networks are kept)
  • πŸ”§ Flexible Stop Modes: Choose between docker compose stop (fast) or down (complete cleanup)

πŸš€ Performance & Reliability

  • ⚑ Parallel Processing: Configurable parallel container operations (1-16 jobs) for faster backups
  • πŸ›‘οΈ Production-Safe: Thread-safe operations with atomic lock protection
  • 🎯 Smart Recovery: Automatic container restart even on backup failures with signal handling
  • πŸ“Š Real-time Monitoring: Live container status tracking with color-coded progress indicators

πŸ’Ύ Advanced Backup Features

  • πŸ”„ rsync-based Synchronization: Standard rsync behavior with intelligent flag validation and multi-tier fallback
  • πŸ” External Encryption: Script creates unencrypted backups. Encryption via external GPG pipelines after backup completion (examples provided)
  • βœ… Backup Verification: Automatic verification of backup integrity and completeness
  • πŸ“ˆ Comprehensive Logging: Detailed logs with ANSI-free output and race-condition-free parallel logging

βš™οΈ Enterprise-Grade Configuration

  • πŸŽ›οΈ Highly Configurable: Extensive command-line options for timeouts, buffers, and behavior
  • πŸ•’ Flexible Scheduling: Perfect for cron automation with various timing options
  • πŸ”’ Security Features: Fail-fast design, input validation, and secure permission handling
  • 🌐 NAS Optimized: Extensively tested on UGREEN NAS (DXP2800). Designed for high compatibility with Synology, QNAP, and other custom Linux NAS systems.

Table of Contents


⚠️ Important Disclaimer

This script is provided "as is" without warranty of any kind. Always test thoroughly in a safe environment and maintain independent backups before production use. The authors assume no responsibility for any data loss, system damage, or service interruption that may result from using this script.

πŸ“‹ Requirements

  • OS: Linux (tested on Ubuntu, Debian, UGREEN NAS DXP2800)
  • Shell: Bash 4.0+
  • Tools: Docker Compose v2 (docker compose), rsync, flock
  • Permissions: sudo access or root execution

⚑ Quick Start (5 Minutes)

1️⃣ One-Line Installation with System Check

πŸ‡ΊπŸ‡Έ English Version:

# Check system requirements first
command -v docker >/dev/null 2>&1 || { echo "❌ Docker not installed. Install Docker first."; exit 1; }
command -v rsync >/dev/null 2>&1 || { echo "❌ rsync not installed. Install: sudo apt install rsync"; exit 1; }
command -v flock >/dev/null 2>&1 || { echo "❌ flock not installed (prevents overlapping backups). Install: sudo apt install util-linux"; exit 1; }
echo "βœ… System requirements met"

# Download and install
wget https://raw.githubusercontent.com/florian101010/NAS-Docker-Backup-rsync/main/docker_backup.sh && \
wget https://raw.githubusercontent.com/florian101010/NAS-Docker-Backup-rsync/main/test_rsync_fix.sh && \
chmod +x docker_backup.sh test_rsync_fix.sh && \
echo "βœ… Installation complete! Next: Test compatibility with ./test_rsync_fix.sh, then configure your paths in the script."

πŸ‡©πŸ‡ͺ German Version:

# Systemvoraussetzungen prΓΌfen
command -v docker >/dev/null 2>&1 || { echo "❌ Docker nicht installiert. Installieren Sie Docker zuerst."; exit 1; }
command -v rsync >/dev/null 2>&1 || { echo "❌ rsync nicht installiert. Installation: sudo apt install rsync"; exit 1; }
command -v flock >/dev/null 2>&1 || { echo "❌ flock nicht installiert (verhindert doppelte Backups). Installation: sudo apt install util-linux"; exit 1; }
echo "βœ… Systemvoraussetzungen erfΓΌllt"

# Download und Installation
wget https://raw.githubusercontent.com/florian101010/NAS-Docker-Backup-rsync/main/docker_backup_de.sh && \
wget https://raw.githubusercontent.com/florian101010/NAS-Docker-Backup-rsync/main/test_rsync_fix_de.sh && \
chmod +x docker_backup_de.sh test_rsync_fix_de.sh && \
echo "βœ… Installation abgeschlossen! Weiter: KompatibilitΓ€t testen mit ./test_rsync_fix_de.sh, dann Pfade im Script konfigurieren."

2️⃣ Configure Your Paths

Open docker_backup.sh with a text editor (e.g., nano) and adjust the following paths to match your system configuration.

⚠️ Important: After saving your changes, always perform a test run with ./docker_backup.sh --dry-run to ensure your configuration is valid.

# Example configuration:
DATA_DIR="/volume1/docker-nas/data"          # Your Docker data directory
STACKS_DIR="/volume1/docker-nas/stacks"      # Your Docker Compose files
BACKUP_SOURCE="/volume1/docker-nas"          # Source directory to backup
BACKUP_DEST="/volume2/backups/docker-nas-backup" # Where to store backups
LOG_DIR="/volume1/docker-nas/logs"           # Log file location

3️⃣ Test & Run

# Test first (safe - no changes made)
./docker_backup.sh --dry-run

# Run interactive backup
./docker_backup.sh

# Automated backup (for cron)
./docker_backup.sh --auto

4️⃣ Next Steps Checklist

After installation, follow these steps in order:

βœ… Immediate Setup (Required):

  1. Test compatibility: ./test_rsync_fix.sh
  2. Configure paths: Edit script with your NAS paths
  3. Test configuration: ./docker_backup.sh --dry-run
  4. First backup: ./docker_backup.sh (interactive)

βš™οΈ Production Setup (Recommended):

  1. Setup automation: Add to cron for daily backups
  2. Test restore: Verify you can restore from backup
  3. Monitor logs: Check backup logs regularly

πŸ”’ Security Setup (Optional):

  1. Preserve ACLs: Use --preserve-acl for file permissions (not encryption)
  2. Secure backup location: Ensure backup destination has proper permissions

🌍 Language Support

Language Script File Status
πŸ‡ΊπŸ‡Έ English docker_backup.sh βœ… Main version
πŸ‡©πŸ‡ͺ German docker_backup_de.sh βœ… Fully translated

πŸ“Š Usage Examples

# πŸ§ͺ Test mode (safe - shows what would happen)
./docker_backup.sh --dry-run

# 🎯 Interactive backup with confirmation
./docker_backup.sh

# πŸ€– Automated backup (perfect for cron)
./docker_backup.sh --auto

# ⚑ High-performance parallel backup
./docker_backup.sh --auto --parallel 4 --use-stop

# πŸ“‹ Backup with ACL preservation (not encryption)
./docker_backup.sh --auto --preserve-acl

πŸ“– Detailed Configuration

πŸ’‘ Pro Tips:

  • Always test with --dry-run first
  • Ensure backup destination has 2x source size available
  • Use --parallel 4 for faster backups on powerful systems
  • Set up cron for automated daily backups

Command Line Options

Option Description Default
--auto Automated execution without confirmation Interactive
--dry-run Test mode without changes Disabled
--parallel N Parallel container operations (1-16) 1
--use-stop Use stop instead of down down
--timeout-stop N Container stop timeout (10-3600s) 60s
--timeout-start N Container start timeout (10-3600s) 120s
--buffer-percent N Storage buffer percentage (10-100%) 20%
--preserve-acl Preserve ACLs and extended attributes (not encryption) Enabled
--skip-backup Only restart containers Disabled
--no-verify Skip backup verification Verification is ON by default

πŸ”„ Automation with Cron

Safe Parallel Cron Examples (v3.5.1+)

# Daily fast backup with parallelization
0 2 * * * /path/to/docker_backup.sh --auto --parallel 4 --use-stop

# Weekly complete backup
0 1 * * 0 /path/to/docker_backup.sh --auto --parallel 2 --preserve-acl

# High-performance setup for large installations
0 2 * * * /path/to/docker_backup.sh --auto --parallel 6 --buffer-percent 25

πŸ›‘οΈ Security Features

Fail-Safe Design

  • Fail-Fast: set -euo pipefail prevents unnoticed errors
  • Signal Handling: Automatic container recovery on interruption (CTRL+C, kill)
  • Input Validation: All parameters validated with range checking
  • Atomic Operations: Lock-protected execution prevents race conditions

Required Dependencies

  • flock: Provides exclusive execution (no overlapping runs) and thread-safe logging when using parallel operations

Backup Verification

  • Directory size comparison with configurable tolerance
  • File and directory count verification
  • ACL and extended attributes support (when available)
  • Detailed error reporting with specific rsync exit code analysis
  • External encryption via GPG pipelines after backup completion (not integrated in script)

πŸ“Š Monitoring & Logging

Log Files

  • Location: /path/to/your/logs/docker_backup_YYYYMMDD_HHMMSS.log
  • ANSI-free output for clean log files
  • Detailed container status with color-coded terminal output
  • Thread-safe logging for parallel operations

Container Status Indicators

  • β–Ά Container started (green)
  • ⏸ Container stopped (yellow)
  • πŸ—‘ Container removed (red)
  • πŸ“¦ Container created (blue)

πŸ”§ Troubleshooting

Common Issues

Missing dependencies:

# flock not found error
sudo apt install util-linux  # Ubuntu/Debian
sudo yum install util-linux  # CentOS/RHEL

Containers won't start:

# Check container status
docker ps -a

# Check specific container logs
docker logs <container_name>

# Manual stack restart
cd /path/to/your/stacks/<stack_name>
sudo docker compose up -d

Backup failures:

# Check available space
df -h /path/to/backup/destination

# Test rsync manually
sudo rsync -av --dry-run /path/to/source/ /path/to/destination/

Permission issues:

# Check backup destination permissions
ls -la /path/to/backup/destination

# Fix permissions if needed
sudo chown -R $(whoami):$(id -gn) /path/to/backup/destination

Dependency Validation

Quick smoke tests:

# Validate flock works (mutex simulation)
LOCK=/tmp/test.lock; exec 9>"$LOCK"; flock -n 9 && echo "βœ… flock OK"

πŸ” Backup Encryption

The script creates unencrypted backups. For encryption, use external GPG pipelines after backup completion as shown below.

Quick Encryption Setup

# 1. Create normal backup
./docker_backup.sh --auto

# 2. Encrypt backup with GPG
tar -czf - /path/to/backup/ | \
gpg --symmetric --cipher-algo AES256 \
> backup_encrypted_$(date +%Y%m%d_%H%M%S).tar.gz.gpg

# 3. Secure password storage for automation
echo "YOUR_SECURE_PASSWORD" | sudo tee /path/to/.backup_password
sudo chmod 600 /path/to/.backup_password

Automated Encrypted Backups

# Cron job for daily encrypted backups
0 2 * * * /path/to/docker_backup.sh --auto && \
tar -czf - /path/to/backup/ | \
gpg --symmetric --cipher-algo AES256 --passphrase-file /path/to/.backup_password \
> /path/to/backup_encrypted_$(date +\%Y\%m\%d_\%H\%M\%S).tar.gz.gpg

Restoring Encrypted Backups

# Decrypt and restore
gpg --decrypt backup_encrypted_YYYYMMDD_HHMMSS.tar.gz.gpg | tar -xzf - -C /

πŸ“– For detailed encryption documentation, see Backup Encryption Guide

🀝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development Setup

git clone https://github.com/florian101010/NAS-Docker-Backup-rsync.git
cd NAS-Docker-Backup-rsync
chmod +x docker_backup.sh test_rsync_fix.sh

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

🎯 Use Cases

Perfect for these scenarios:

  • 🏠 Home Labs: Protect your self-hosted services (Plex, Nextcloud, etc.)
  • 🏒 Small Business: Backup critical Docker applications safely
  • πŸ”§ Development: Consistent backup of development environments
  • πŸ“Š Production: Reliable backup for production Docker stacks

πŸ™ Acknowledgments

  • πŸ› οΈ Built on rsync: Powered by the robust rsync project for reliable file synchronization
  • 🐳 Docker Integration: Leverages Docker and Docker Compose ecosystem
  • βœ… Tested & Optimized: UGREEN NAS DXP2800
  • 🌟 Open Source: MIT licensed for maximum flexibility

πŸ“ˆ Version History

See CHANGELOG.md for detailed release notes.

πŸ“š Documentation

Quick Start

Detailed Guides

  • πŸ‡ΊπŸ‡Έ English Manual - Complete user guide in English
  • πŸ‡©πŸ‡ͺ German Manual - VollstΓ€ndige Anleitung auf Deutsch

Automation

Development



πŸ“Έ Screenshots

Backup Process in Action

Screenshot showing the script's initialization and container stop process Screenshot showing the rsync backup process and verification steps Screenshot showing the container start process and final summary report

About

Docker backup script for NAS systems with parallelization, error handling, and automatic container management. Uses rsync for data synchronization, supports ACL preservation, and includes fallback mechanisms for broad compatibility.

Topics

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Packages

No packages published

Languages