📰 SmartContentPipelineAi Project Backend CDK

Welcome to the SmartContentPipelineAi Project Backend! This project sets up a serverless infrastructure for the SmartContentPipelineAi project using AWS CDK (Cloud Development Kit). The stack includes S3 for data storage, Lambda functions for processing, and API Gateway for exposing APIs. This enables automatic deployment, secure data handling, and API integrations.

🚀 Prerequisites

Before proceeding, ensure you have the following installed:

• AWS CLI 🌐
• Node.js 🛠️
• Python 3.7+ 🐍
• AWS CDK 🌍

📂 Project Structure

• S3 Bucket: Used for storing and managing data.
• Lambda Functions: Execute serverless functions to handle different tasks.
• API Gateway: Exposes endpoints for external communication.
• IAM Roles: Provides necessary permissions for Lambda functions.

📊 Architecture Diagram:

⚙️ Setup Instructions

Install AWS CDK 🛠️

To install AWS CDK, ensure you have Node.js (with NPM), AWS CLI configured, and then run

npm install -g aws-cdk

Clone the Project 🧑‍💻

First, clone the repository from GitHub:

git clone https://github.com/TejaTalachiru/smartcontent-pipeline-ai.git
cd backend

Setup Virtual Environment 🌐

To create and activate a virtual environment (Mac/Linux):

python -m venv .venv
source .venv/bin/activate

For Windows:

python -m venv .venv
.venv\Scripts\activate.bat

Install Required Dependencies 📦

Install the Python dependencies using the following command:

pip install -r requirements.txt

Update Bucket Name in cdk.json 📑

You need to specify the S3 bucket name in the cdk.json file. Open the file and update the value for bucket_name:

{
  "app": "python3 app.py",
  "context": {
    "bucket_name": "<your-unique-bucket-name>"
  }
}

Login to AWS CLI 🔐

Ensure you are logged in to your AWS account:

aws configure

You will need to input your AWS Access Key, Secret Key, and default region.

Bootstrap the CDK 🚀

Before deploying the stack, you need to bootstrap your environment:

cdk bootstrap

Synthesize CloudFormation Template 🏗️

To generate the CloudFormation template for this project:

cdk synth

Deploy the Stack 🚀

To deploy the stack to your AWS environment:

cdk deploy

This will deploy the S3 bucket, Lambda functions, API Gateway, and IAM roles.

🛠️ Lambda Functions

This CDK project deploys multiple Lambda functions:

• SummaryLambda: Handles requests for summary creation.
• TitleLambda: Handles requests for title generation.
• UpdateDataLambda: Manages data updates.
• GetDataLambda: Fetches data from S3.
• GeneratePresignedUrlNewLambda: Generates presigned URLs for new uploads.
• GeneratePresignedUrlExistingLambda: Generates presigned URLs for existing objects.
• GetPresignedUrlLambda: Retrieves presigned URLs.
• TranslationLayer: To generate the IDML.

🌐 API Endpoints

Once the stack is successfully deployed, you will see the API URL in the deployment outputs. With this URL, you can access the following API endpoints:

• Summary API (POST): /summary
• Title API (POST): /title
• Update Data API (POST): /update_data
• Get Data API (GET): /get_data
• Generate Presigned URL for New Uploads (GET): /generate_presigned_url_new
• Generate Presigned URL for Existing Files (GET): /generate_presigned_url_existing
• Get Presigned URL (GET): /get_presigned_url
• Idml Generation (GET): /idml_gen

🧹 Cleanup

To delete the resources and avoid incurring charges, run:

cdk destroy

🛠️ Useful CDK Commands

• cdk ls: List all stacks in the app.
• cdk synth: Emit the synthesized CloudFormation template.
• cdk deploy: Deploy this stack to your AWS account/region.
• cdk diff: Compare the deployed stack with the current state.
• cdk docs: Open the CDK documentation.

❓Troubleshooting

If you encounter any issues during the setup or deployment process, try the following steps:

Check AWS CLI Configuration: Ensure your AWS CLI is configured correctly by running:

aws configure

Confirm that the region, access key, and secret key are correctly set.

Virtual Environment Issues: If you encounter issues with the virtual environment (e.g., packages not found), try recreating the environment:

deactivate  # If the virtualenv is currently active
rm -rf .venv  # Delete the existing virtual environment
python -m venv .venv  # Recreate the virtual environment
source .venv/bin/activate  # Activate on MacOS/Linux
.venv\Scripts\activate.bat  # Activate on Windows
pip install -r requirements.txt  # Reinstall dependencies

Deployment Fails 🚨:

• Ensure that the bucket name in cdk.json is unique.
• Check CloudFormation in the AWS Console for any specific error messages.
• Validate that your AWS credentials have the necessary permissions to create resources.

Common Errors 🧑‍💻 :

• S3 Bucket Already Exists: Ensure your bucket name is globally unique in the cdk.json file.
• Ensure you have necessary IAM roles and permissions to deploy the Stack.

Helpful Links:

• AWS CDK Documentation 🌐
• AWS Lambda Documentation 🛠️
• AWS S3 Documentation 📦
• AWS API Gateway Documentation 🌐