Skip to content

rug-ds-lab/planning-dynamic-spark-supplemental

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Supplemental files for "Adaptive On-the-fly Changes in Distributed Processing Pipelines"

Supplimental experiment files for the paper "Adaptive On-the-fly Changes in Distributed Processing Pipelines". This code depends on the dynamic-spark-core and dynamic-spark-planning libraries, which are at this point not yet open source. Experiments should be compiled with the SBT Assembly plugin and submitted to a standalone Spark cluster using the scripts in scripts/run (see all-experiments_cluster.sh).

DAG representation of the scenarios

We have also included a techincal DAG view of the pipelines that show the Spark representation, the assigned modules and the constraints set on the pipeline.

Simple scenario

Simple DAG

Middle scenario

Middle DAG

Complex scenario

Complex DAG

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published