Build clickstream analytics on AWS for your mobile and web applications

🌳 A sustainable Terraform Package which creates resources for Messaging Services (EventBridge, MSK, SNS, SQS) on AWS

Oracle database CDC (Change Data Capture)

使用AWS托管服务的最佳实践样例

Terraform module to create AWS MSK (Managed Streaming for Kafka) resources 🇺🇦

Use-Case: Airline on-time performance

Oracle audit files to Apache Kafka/Amazon MKS/Amazon Kinesis transfer

Oracle Database Automatic Diagnostic Repository messages (alert.log & listener.log) to Apache Kafka or Amazon Kinesis transfer

The CloudFormation Resource Provider Package For Amazon MSK Connect

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK Serverless and MSK Connect (Debezium)

This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (MSK) and MSK Serverless into Apache Iceberg table in S3 with AWS Glue Streaming.

Terraform Kafka Provider

Streaming data pipeline to continuously load data from an Amazon MSK or MSK Serverless cluster to Amazon S3 using Amazon Kinesis Data Firehose.

Terraform module to create kafka resource on AWS. AWS MSK (Managed Streaming for Apache Kafka) is a fully managed service that simplifies the deployment, management, and operation of Apache Kafka clusters. Apache Kafka is an open-source distributed streaming platform used for building real-time streaming data p

x-msk_cluster ECS Compose-X extension package

Pinterest's experiment analytics data pipeline which runs thousands of experiments per day and crunches billions of datapoints to provide valuable insights to improve the product.

A Cloud based Reddit stock sentiment analyzer that analyzes overall sentiment from a configurable selection of stock subreddits for each stock. The architecture utilizes AWS MSK (Kafka), AWS EMR (PySpark) and AWS Lambda (Python 3) for maximum scalability and the OpenAI API for sentiment analysis through prompt engineering.

Pinterest data pipeline - designing an end-to-end pipeline utilising AWS cloud technologies and Databricks for analysing real-time and historical pinterest-emulated data.

My AWS Playground

Pinterest's experiment analytics data pipeline which runs thousands of experiments per day and crunches billions of datapoints to provide valuable insights to improve the product.