Skip to content

Commit

Permalink
Merge pull request #7 from yvonnegitau/feature/dagster
Browse files Browse the repository at this point in the history
Fixing Docuementation
  • Loading branch information
yvonnegitau authored Feb 11, 2025
2 parents 2ddd5f5 + f281319 commit ef21108
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions docs/orchestration.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Data Orchestration

For data orchestration we will be using Dagster. We will add Dagster to our docker compose. When running Dagster, one needs the below long running containers :

* Webserver
* Daemon
* Code Location (each code has its own container)
* Metadata database
- Webserver
- Daemon
- Code Location (each code has its own container)
- Metadata database

Each run will be executed on its own container. Webserver and Daemon will have the same Docker image. We will also have files below which are a requirement:

Expand All @@ -17,6 +18,7 @@ Dagster OSS does not come with authentication. I have added basic authentication
**NOTE: Sessions is not handled well. A better authentication should be used prod. This auth also does not handle RBAC. This only secures this setup.**

## Run Code

`docker compose up`

The webserver can be accessed through `localhost:3000`

0 comments on commit ef21108

Please sign in to comment.