GTFS schedule ingestion into postgres mapreduce

[kylerchin]

Some ideas about this:

• MapReduce style: A single instance holds leader position, listens for other nodes on the cluster. 4 times a day at 00:00, 06:00, 12:00 and 18:00, the master assigns out feed ids to be downloaded processed by each server. Assignment is done via zookeeper and RPC via tarpc.

I still must devise a method for fault tolerance and schema upgrades. Also how do the nodes find each other aka service discovery, so they can talk to each other and exchange RPC calls.