Home

Welcome to the data-engineering-devs wiki!

This contains documentation for different technologies - these documents will be updated regularly, with new versions announced on the Slack 'de_issues' channel.

Data Engineering Ecosystem

AWS

• AWS @ Insight
• Adding Volume to AWS EC2 Instances

Ingestion

• Kafka
• Kafka - Advanced(kafka-advanced)
• Camus
• Secor

File Systems

• HDFS
• [AWS S3] (aws-simple-storage-service-(s3))
• Alluxio

Batch Processing

• Hadoop Intro
• Hadoop MapReduce
• Hive
• Pig
• Spark Intro
• Presto

Stream Processing

• Spark Streaming Dev
• Storm Dev

Databases

• Cassandra Dev
• HBase Dev
• Elasticsearch Dev
• Redis Dev

Web Framework

• Flask

Others

• Zookeeper
• Supervisor
• Github Intro
• CDH5