Data Engineering ETL Google Cloud Platform Project

An end-to-end modern data engineering project, including deployment of ETL pipeline on Google Cloud Platform, using BigQuery for data analysis and leveraging Looker to generate an insight dashboard.

Technology Stack

Languages:

• Python
• SQL

Google Cloud Platform:

• Google Storage
• Google Engine
• Big Query
• Looker Studio

Modern Data Pipeline Tool:

• Mage - https://www.mage.ai

Data Source

The dataset is provided by TLC Trip Record Data, including yellow and green taxi trip records. It includes fields capturing pick-up and drop-off dates/times, pick-up and drop-off locations, trip distances, itemized fares, rate types, payment types, and driver-reported passenger counts.

Data source link: https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page

Data Dictionary: https://www.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf

Data Modeling

ETL Pipeline

Looker Dashboard

Uber_Dashboard_Present.mov

Contact

Adad Al Shabab(sababadad74@gmail.com)