AZTK powered by Azure Batch: On-demand, Dockerized, Spark Jobs on Azure
-
Updated
Feb 2, 2021 - Python
AZTK powered by Azure Batch: On-demand, Dockerized, Spark Jobs on Azure
Experiments produced during an end of studies project (ETS, H2018)
Airflow, Spark and Kafka example
Vagrant box to run Spark jobs and unit testing with PySpark
A Python framework for managing Dataproc cluster and Scheduling PySpark Jobs over it. Additionally it provides docker based development for debugging PySpark jobs.
Big data platform - deprecated
A Python library for sending notifications on the current status of a Spark Job.
SPARK jobs both in batch processing and streaming.
Add a description, image, and links to the spark-jobs topic page so that developers can more easily learn about it.
To associate your repository with the spark-jobs topic, visit your repo's landing page and select "manage topics."