Introduction to Apache Spark

Kevin Mader (4Quant and ETH Zurich)

The presentation

The tutorial was given as part of the Big Data course by Professor Thomas Hofmann at ETH Zurich (www.da.inf.ethz.ch/lectures/bigdata/)

Learn more at

• Amazon Cloud
• Spark Official EC2 Scripts
• AMP Lab Big Data Mini-Course
• Applied Natural Language Processing,
• Spark on AWS 1
• Spark on AWS 2
• Using HPC Clusters
• Setting up Spark on top of Sun Grid Engine (PSI) - https://github.com/4Quant/sge_spark
• Setting up Spark on the Brutus and Euler Clusters (ETH) - https://bitbucket.org/skicavs/spark-on-brutus
• Spark Demo from this presentation - https://gist.github.com/kmader/755c2d99c23f4cbe2e74

Imaging with Spark

• 4Quant: From Images to Statistics - http://www.4quant.com (@4Quant)
• X-Ray Imaging Group at ETH Zurich - http://bit.ly/1gD8wKb
• Quantitative Big Imaging Course at ETH Zurich - http://bit.ly/1kj9mnq
• Presentation at Spark Summit 2014 - https://rawgit.com/4Quant/spark-summit-2014-presentation/master/ssPresentation.html
• Spark Introduction (http://4quant.github.io/spark-introduction/)

Graphs on Spark

• FEM Demo on Gist - https://gist.github.com/kmader/6456262935af381c8dbe