Skip to content

Latest commit

 

History

History
374 lines (290 loc) · 11.8 KB

README.md

File metadata and controls

374 lines (290 loc) · 11.8 KB

MongoDB Database Replication & Anonymization

Automated infrastructure for deploying MongoDB on AWS EC2 with EBS volumes, and safely cloning production data to staging environments with PII anonymization.

Terraform CI Terraform CD Prod to Staging Sync Cleanup Resources

📅 Last Restore Status

Environment Last Restored Status Documents Anonymized Duration
Staging 2026-03-03 10:51 UTC ✅ Success 10 ✅ Yes 2m 56s

💡 Tip: This table is automatically updated after each successful sync using the GitHub Actions workflow.

🎯 Overview

This project provides:

  • Terraform Infrastructure: Deploy MongoDB on EC2 with dedicated EBS data volumes
  • GitHub Actions Automation: Clone production MongoDB volumes to staging
  • Data Anonymization: Automatically anonymize PII data in staging
  • Resource Management: Track and cleanup old snapshots/volumes
  • CI/CD Ready: Automated workflows with self-hosted runners

📁 Project Structure

.
├── terraform/                  # Infrastructure as Code
│   ├── modules/
│   │   └── ec2/               # EC2 + EBS module for MongoDB
│   └── stacks/
│       ├── production/        # Production environment
│       └── staging/           # Staging environment
├── mongodb/                   # Database scripts
│   ├── setup_database.js     # Create DB with mock PII data
│   ├── anonymize_data.js     # Simple anonymization
│   ├── anonymize_with_hash.js # Hash-based anonymization
│   └── restore_original_data.js
├── .cleanup/                  # Resource tracking
│   ├── resource-tracker.json # Tracked snapshots/volumes
│   └── README.md            # Cleanup documentation
└── .github/
    └── workflows/
        ├── prod-to-staging-sync.yml      # Production to staging sync
        ├── cleanup-resources.yml         # Automated resource cleanup
        ├── README-SYNC.md               # Workflow documentation
        └── QUICKSTART-SYNC.md           # Quick start guide

🚀 Quick Start

1. Deploy Infrastructure with Terraform

# Deploy staging environment
cd terraform/stacks/staging
terraform init
terraform plan
terraform apply

# Deploy production environment
cd ../production
terraform init
terraform apply

What gets created:

  • EC2 instance with Amazon Linux 2023
  • Root volume (8GB) for OS
  • Data volume (20GB) for MongoDB
  • Security groups for SSH and MongoDB access
  • MongoDB 7.0 installed and configured

2. Setup Initial Database

# SSH into production instance
ssh ec2-user@<production-ip>

# Copy and run the setup script
mongosh < setup_database.js

3. Clone Production to Staging

Using GitHub Actions Workflow:

  1. Add AWS credentials to GitHub Secrets (AWS_SECRET_ACCESS_ID, AWS_SECRET_ACCESS_KEY)
  2. Go to Actions → "Production to Staging DB Sync"
  3. Click "Run workflow"
  4. Select anonymization option (default: enabled)
  5. Monitor progress through 9 separate jobs

📖 Quick Start | Full Docs

What happens during sync:

  • ✅ Validates AWS resources
  • ✅ Stops staging MongoDB
  • ✅ Creates snapshot from production
  • ✅ Swaps EBS volumes
  • ✅ Mounts new volume
  • ✅ Starts MongoDB
  • ✅ Anonymizes PII data (optional)
  • ✅ Tracks old resources for cleanup
  • ✅ Updates README with sync status

📋 Features

Infrastructure (Terraform)

  • ✅ EC2 instances with MongoDB 7.0
  • ✅ Separate EBS volumes for data
  • ✅ Auto-mounting and configuration via user_data
  • ✅ Security groups with proper access controls
  • ✅ Support for multiple environments (staging/production)
  • ✅ XFS filesystem (MongoDB recommended)

Automation (GitHub Actions)

  • ✅ Automated EBS snapshot creation
  • ✅ Volume-based replication
  • ✅ Zero-downtime for production
  • ✅ Automatic volume swap
  • ✅ Self-hosted runner on staging
  • ✅ Built-in data anonymization
  • ✅ Resource tracking for cleanup
  • ✅ Auto-update README status

Data Management

  • ✅ Mock PII data generation
  • ✅ Simple anonymization (User 1, User 2, etc.)
  • ✅ Hash-based anonymization (non-reversible)
  • ✅ Data restoration scripts
  • ✅ 10 sample users with realistic PII

🔐 Security & Compliance

PII Data Protection

The project includes two anonymization strategies:

Simple Anonymization:

  • Names → "User [ID]"
  • Email → "user[ID]@anonymized.local"
  • SSN → "XXX-XX-[ID]"
  • Address → Redacted values

Hash-Based Anonymization:

  • One-way hash functions
  • Non-reversible transformation
  • Maintains referential consistency
  • Suitable for production-like testing

AWS Security

  • EC2 security groups restrict access
  • EBS volumes encrypted at rest
  • SSM for secure command execution
  • IAM roles with least privilege
  • Snapshots properly tagged

📖 Documentation

🔄 GitHub Actions Workflows

Production to Staging Sync

File: prod-to-staging-sync.yml

✨ Features:

  • 9 separate jobs for granular control
  • Self-hosted runner on staging EC2
  • GitHub-hosted runner for AWS API calls
  • Detailed per-job summaries
  • Maximum visibility and debugging
  • Fail-fast at job level
  • ~3-10 minutes duration
  • Automatic README updates
  • Resource tracking for safe cleanup

📊 Jobs:

  1. Setup & Validation - Discover volume IDs
  2. Stop MongoDB - Stop service on staging
  3. Create Snapshot - Snapshot production volume
  4. Swap Volumes - Create and attach new volume
  5. Mount Volume - Mount on staging EC2
  6. Start & Verify - Start MongoDB and verify data
  7. Anonymize Data - Optional PII anonymization
  8. Track Resources - Track snapshot/volume for cleanup
  9. Final Summary - Update README and report

Automated Resource Cleanup

File: cleanup-resources.yml

✨ Features:

  • Scheduled daily cleanup (2 AM UTC)
  • Manual trigger with dry-run option
  • Age-based deletion (default: 1 day)
  • Tracks all snapshots and volumes
  • Safe rollback capability
  • Automatic tracker file updates

Cleanup Options:

  • Scheduled: Runs daily at 2 AM UTC, deletes resources older than 1 day
  • Manual: Run on-demand with custom age threshold and dry-run preview

Workflow Benefits

Feature Description
Zero Production Impact Works on snapshots, production untouched
Self-Hosted Execution MongoDB operations run on staging EC2
Resource Management Track and cleanup old snapshots/volumes
Safety First Dry-run mode, rollback instructions
Cost Efficient Automated cleanup prevents AWS cost buildup
Visibility Per-job logs and comprehensive summaries
Automation Scheduled cleanup, auto-update README

🛠️ Prerequisites

For Terraform

  • Terraform 1.0+
  • AWS CLI configured
  • AWS credentials with EC2/VPC permissions

For GitHub Actions Workflows

  • AWS credentials (stored as GitHub Secrets: AWS_SECRET_ACCESS_ID, AWS_SECRET_ACCESS_KEY)
  • Self-hosted runner configured on staging EC2
  • IAM permissions for EC2, EBS, Snapshots
  • GitHub repository access to Actions

🔄 Typical Workflow

  1. Initial Setup: Deploy infrastructure with Terraform
  2. Populate Production: Load production data
  3. Clone to Staging: Use GitHub Actions workflow to sync and anonymize
  4. Test in Staging: Verify functionality with anonymized data
  5. Cleanup Resources: Run cleanup workflow to delete old snapshots/volumes
  6. Repeat: Run sync as needed (on-demand or scheduled)

💰 Cost Estimate

Per Environment (Monthly):

  • EC2 t3.large: ~$60/month
  • EBS Root (8GB gp3): ~$0.64/month
  • EBS Data (20GB gp3): ~$1.60/month
  • Data transfer: Free (same region)
  • Total: ~$62/month per environment

Snapshots:

  • ~$0.05/GB/month (incremental)
  • 20GB snapshot: ~$1/month

🎯 Use Cases

  • Development/Testing: Safe staging environment with anonymized data
  • Compliance: Meet GDPR/CCPA requirements for test data
  • Disaster Recovery: Practice restoration procedures
  • Performance Testing: Use production-sized datasets
  • Training: Onboard new team members safely

🔧 Configuration

Terraform Variables

Edit terraform/stacks/staging/terraform.tfvars:

instance_type = "t3.large"
root_volume_size = 8
mongodb_data_volume_size = 20

Workflow Configuration

Edit workflow environment variables in .github/workflows/prod-to-staging-sync.yml:

env:
  AWS_REGION: us-west-2
  PROD_INSTANCE_ID: i-0e360e7615a63a796
  STAGING_INSTANCE_ID: i-05661b198eb8d9b0a

Cleanup Schedule

Edit cron schedule in .github/workflows/cleanup-resources.yml:

schedule:
  - cron: '0 2 * * *'  # Daily at 2 AM UTC

📊 Monitoring

Check MongoDB status:

sudo systemctl status mongod

View data volume:

df -h | grep mongodb

Count documents:

mongosh --eval "use userdb; db.users.countDocuments()"

🐛 Troubleshooting

Common Issues

Workflow fails to find volumes:

  • Verify instance IDs in workflow env variables
  • Check that volumes are attached to /dev/sdf
  • Ensure AWS credentials have EC2 describe permissions

Self-hosted runner offline:

  • SSH to staging EC2 and check runner status
  • Restart runner service if needed
  • Verify GitHub runner token hasn't expired

Cleanup finds 0 resources:

  • Check max_age_days setting (use 0 for same-day cleanup)
  • Verify .cleanup/resource-tracker.json has entries
  • Date comparison uses <= so same-day resources are included

README not updating:

  • Check final-summary job logs
  • Verify git push succeeded (check for conflicts)
  • Ensure workflow has contents: write permission

See detailed documentation:

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Test thoroughly
  5. Submit a pull request

📄 License

This project is provided as-is for educational and internal use purposes.

🙋 Support

For issues or questions:

  1. Check the documentation in ansible/README.md
  2. Review troubleshooting guides
  3. Check AWS CloudWatch logs
  4. Verify IAM permissions

🎓 Learning Resources