Skip to content

alvansaid/airflow-with-spark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Airflow with Pyspark inside

  • build docker using docker docker build -t airflow-spark .
  • run docker compose using docker compose -f airflow.yaml up

Running the python script on Airflow

  • run the script by sudo -u airflow python /opt/airflow/scripts/script.py

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published