A quick start repo for creating Airflow dags to manage your dbt Cloud jobs.
Please see the dbt docs on Airflow and dbt Cloud for an in-depth step-by-step tutorial. Only the key concepts are included in this README.
Please see the dbt Cloud Airflow Provider docs for the most up-to-date details on the Airflow providers used in the example dags of this repo.
To use this repo you'll need:
- dbt Cloud Teams or Enterprise account (with admin access) in order to create a service token.
- Docker Desktop installed
- Astro CLI installed. On Mac use
brew install astro.
After creating all the requisite accounts and installing Astro CLI, simply:
- Clone this repo locally:
gh repo clone dbt-labs/airflow-dbt-cloud - Create a dbt Cloud service token
- Create a dbt Cloud job
- Start your Astro container:
astro dev start - Create a new dbt Cloud Airflow connection. To do this:
- From the Airflow interface, navigate to Admin and click on Connections
- Click on the + sign to add a new connection, then click on the drop down to search for the dbt Cloud Connection Type
- Add in your connection details and your default dbt Cloud account id. This is found in your dbt Cloud URL after the accounts route section (
/accounts/{YOUR_ACCOUNT_ID})
- Update the code in the dags/ folder to use your Airflow connection, dbt account and dbt job information
- Run your first dbt Cloud Airflow DAG 🎉