This repo is is intended as a starter guide for using pyspark in development and production. It is initially created for the Cairo Data Science Meetup introductory talk to spark

The content of this repo is as follows (note that vms and boostrap folders contain their respective readme files):

• presentations contains the slides for the talk, and the script for the content that was presented in the meetup.
• vms contains the configuration files and descritpion of how to run a Spark virtual machine locally using vagrant.
• boostrap contains the scripts needed as boostrap actions when running a Spark1.4 cluster on amazon through EMR.

To learn more about Spark, here are favourite sources:

• Spark Programming Guide
• PySpark documentaion
• edX courses for Spark:
  1. https://courses.edx.org/courses/BerkeleyX/CS100.1x/1T2015/info
  2. https://courses.edx.org/courses/BerkeleyX/CS190.1x/1T2015/info