This project demonstrates key PySpark performance optimization techniques using a synthetic banking transactions dataset (~5,000 records). Built using Databricks and Delta Lake.

PySpark ETL & analytics pipeline for taxi trip ETA, partitioned Parquet, windowed aggregations and performance patterns.

This repository showcases how to setup a Scala Spark job on Docker and in Dataproc and execute a Broadcast Join technique.

🚖 Ingest and analyze NYC yellow taxi data with a streamlined ETL pipeline, featuring data cleaning, analytics, and business-ready outputs.