Data Encryption at Rest and in Transit

[rucka]

Story Statement

As a security officer
I want all organizational KB content encrypted at rest and in transit
So that our data is protected according to enterprise security standards even if storage or network is compromised

Where: Knowledge service infrastructure — encryption layer across storage and transport

Epic Context

Parent Epic: Platform Hardening & Enterprise Readiness #68
Status: Refined
Priority: P0 (Must-Have)

Status Workflow

• Refined: Story is detailed, estimated, and ready for development
• In Progress: Story is actively being developed
• Done: Story delivered and accepted

Acceptance Criteria

Functional Requirements

1. Given a KB package is uploaded to S3
   When it is stored
   Then it is encrypted with AES-256 via S3 server-side encryption (SSE-KMS) using org-scoped KMS key

2. Given the PostgreSQL database stores org data
   When sensitive fields are written (API key hashes, user emails, org settings)
   Then they are encrypted at rest via database-level TDE or application-level encryption

3. Given any client communicates with the service
   When a request is made
   Then TLS 1.2+ is enforced (TLS 1.3 preferred); HTTP requests are rejected or redirected to HTTPS

4. Given a security officer checks encryption status
   When they call GET /api/v1/organizations/acme/encryption-status
   Then the service returns: { "storage_encryption": "AES-256-KMS", "transit_encryption": "TLS-1.3", "key_rotation": { "last_rotated": "...", "next_rotation": "..." }, "db_encryption": "active" }

5. Given encryption keys are managed via KMS
   When automatic key rotation is due (every 90 days)
   Then KMS rotates the key; existing data remains accessible (envelope encryption); new writes use new key; audit log records rotation event

6. Given TLS certificates
   When certificate is within 30 days of expiry
   Then auto-renewal triggers (Let's Encrypt or ACM); alert fires if renewal fails

Business Rules

• S3 encryption: SSE-KMS with per-org or shared KMS key (AES-256)
• DB encryption: TDE (if managed DB supports) or application-level field encryption for sensitive data
• Transit: TLS 1.2 minimum, TLS 1.3 preferred; HSTS header enforced
• Key rotation: automatic every 90 days via KMS; envelope encryption ensures zero-downtime rotation
• Certificate management: auto-renewal via Let's Encrypt (self-hosted) or ACM (AWS)
• All key management operations logged to audit trail

Edge Cases and Error Handling

• KMS unavailable: Service degrades gracefully — reads succeed (cached key), writes queue until KMS recovers; alert fires
• Certificate renewal failure: Alert fires 14 days before expiry; ops can manually renew
• TLS downgrade attack: Strict transport security (HSTS) prevents downgrade; minimum TLS 1.2 enforced at load balancer
• Key rotation during heavy write load: Envelope encryption ensures new key is used for new writes; no re-encryption needed

Definition of Done Checklist

Development Completion

• All 6 acceptance criteria implemented and verified
• S3 SSE-KMS encryption configured
• DB encryption (TDE or application-level) active
• TLS 1.2+ enforced with HSTS
• Encryption status endpoint
• KMS key rotation automation
• Certificate auto-renewal configured
• Integration tests for encryption verification

Quality Assurance

• Stored data verified encrypted at rest (S3 metadata, DB inspection)
• TLS 1.0/1.1 connections rejected
• Key rotation doesn't cause data access failures
• HSTS header present on all responses

Deployment and Release

• KMS setup documented per cloud provider
• Certificate management procedure documented
• Encryption status endpoint accessible to security officers

Story Sizing and Sprint Readiness

Refined Story Points

Final Story Points: XL(8)
Confidence Level: Medium
Sizing Justification: Mostly infrastructure configuration (S3 SSE, TLS, DB encryption) rather than application code. Key rotation and encryption status endpoint are the main application-level work. Complexity varies by hosting platform.

Sprint Capacity Validation

Sprint Fit Assessment: Fits in single sprint
Total Effort Assessment: Yes

Dependencies and Coordination

Story Dependencies

Prerequisite Stories: Epic #66 #149 (Org Setup — service infrastructure), #150 (Upload — S3 storage)
Dependent Stories: #167 (Backup — encrypted backups)

External Dependencies

Infrastructure Requirements: KMS service (AWS KMS, GCP KMS, or HashiCorp Vault), TLS certificate provider

Validation and Testing Strategy

Acceptance Testing Approach

Testing Methods: Infrastructure verification tests (S3 encryption metadata, TLS certificate check); integration tests for encryption status endpoint; penetration test for TLS enforcement
Test Data Requirements: Test KMS key, test certificate
Environment Requirements: KMS test instance (localstack KMS or dev KMS)

Notes

Refinement Insights: Most work is infrastructure configuration. Application-level work is limited to encryption status endpoint and sensitive field encryption. Record KMS choice as ADR.

Technical Analysis

Implementation Approach

Technical Strategy: Infrastructure-first: configure S3 SSE-KMS at bucket level (all objects encrypted automatically). Configure DB TDE (if RDS/managed) or use application-level encryption with crypto.createCipheriv for sensitive fields. TLS termination at load balancer/reverse proxy (nginx/ALB). Application: encryption status endpoint reads infrastructure state.
Key Components: S3 encryption config, DB encryption config/module, TLS termination config, encryption status endpoint, KMS rotation config
Data Flow: Upload → S3 SSE-KMS (automatic) | DB write → field encryption (if app-level) | All requests → TLS termination at LB

Technical Requirements

• S3: ServerSideEncryption: 'aws:kms' + SSEKMSKeyId in bucket policy (enforced for all PutObject)
• DB: pgcrypto extension for sensitive field encryption or application-level aes-256-gcm
• TLS: nginx/ALB config with ssl_protocols TLSv1.2 TLSv1.3; + HSTS max-age=31536000; includeSubDomains
• KMS: auto-rotation policy (90 days); envelope encryption (data key encrypted by master key)
• Encryption status: query S3 bucket encryption config + DB encryption status + TLS cert expiry

Technical Risks and Mitigation

Risk	Impact	Probability	Mitigation Strategy
Application-level DB encryption performance overhead	Medium	Medium	Encrypt only sensitive fields (PII, keys); benchmark impact
KMS vendor lock-in	Medium	Low	Abstract KMS interface; support multiple providers

Spike Requirements

Required Spikes: None — patterns well-understood; KMS provider decision recorded as part of infrastructure ADR