Feature opensearch integration

DEPLOYMENT_REPORT.md

@@ -0,0 +1,101 @@
+# ProactivePulse AI AWS Deployment Report
+
+## Deployment Status
+✅ **Script Updated Successfully**
+
+The deployment script has been updated to focus only on database resources and ensure all tables have the required Global Secondary Indexes (GSIs).
+
+## What the Script Now Does
+
+### 1. ✅ S3 Buckets
+- Creates two S3 buckets for raw and processed data
+- Checks for existing buckets to prevent conflicts
+
+### 2. ✅ DynamoDB Tables with GSIs
+- Checks existing DynamoDB tables and adds missing Global Secondary Indexes:
+  - **PulseAI-Insights** table with `status-created_at-index` GSI
+  - **PulseAI-Anomalies** table with `metric-timestamp-index` GSI
+  - **PulseAI-Clusters** table with `category-created_at-index` GSI
+- Creates tables with proper GSIs if they don't exist
+- Uses PAY_PER_REQUEST billing mode for cost efficiency
+
+### 3. ❌ EC2 Infrastructure (Removed)
+- Removed EC2 deployment as it's not needed for this deployment
+- Focus is now purely on database resources
+
+## Current Status Check
+
+Based on our checks, the following tables exist but may be missing their required GSIs:
+- `pulseai-Insights`
+- `pulseai-Anomalies`
+- `pulseai-Clusters`
+
+## Manual Setup Required
+
+### 1. ⚠️ OpenSearch Domain
+The deployment script **does not create** the OpenSearch domain due to complexity and security considerations. You must manually:
+
+1. Go to AWS OpenSearch Service console
+2. Create a new domain with name: `pulseai-search`
+3. Configure with appropriate instance type and storage:
+   - For development: `t3.small.search` (2 vCPU, 2 GB RAM)
+   - For production: `r6g.large.search` or larger
+4. Set up access policy to allow access from your VPC or IP
+5. Note the domain endpoint for configuration
+
+### 2. ⚠️ Environment Configuration
+After creating the OpenSearch domain, configure your environment variables:
+
+```bash
+OPENSEARCH_ENDPOINT=https://your-domain.us-east-1.es.amazonaws.com
+OPENSEARCH_INDEX_PREFIX=pulseai
+OPENSEARCH_USE_SSL=true
+OPENSEARCH_VERIFY_CERTS=true
+```
+
+## Security Considerations
+
+1. ✅ No hardcoded credentials - uses AWS SSO profile
+2. ✅ Proper IAM permissions required for execution
+3. ⚠️ Configure proper access policies for OpenSearch domain
+
+## Performance Benefits
+
+With the updated script, your application will now benefit from:
+
+1. **Faster Queries**: GSIs enable efficient querying patterns:
+   - Query insights by status and time range
+   - Query anomalies by metric and time range
+   - Query clusters by category and time range
+
+2. **Cost Efficiency**: PAY_PER_REQUEST billing mode for DynamoDB
+
+3. **Scalability**: Proper indexing for handling larger datasets
+
+## How to Run the Updated Script
+
+1. Ensure AWS SSO is configured:
+   ```bash
+   aws configure sso --profile proactive-pulse
+   ```
+
+2. Run the deployment script:
+   ```bash
+   cd deployment
+   deploy-aws.bat
+   ```
+
+## Next Steps
+
+1. Run the updated deployment script to add missing GSIs
+2. Manually create the OpenSearch domain
+3. Configure environment variables with OpenSearch endpoint
+4. Test the application
+
+## Troubleshooting
+
+If you encounter issues:
+
+1. Verify AWS credentials: `aws sts get-caller-identity --profile proactive-pulse --region us-east-1`
+2. Check table creation: `aws dynamodb list-tables --profile proactive-pulse --region us-east-1`
+3. Verify GSI creation: `aws dynamodb describe-table --table-name pulseai-Insights --profile proactive-pulse --region us-east-1 --query "Table.GlobalSecondaryIndexes"`