A Python program for distributed (client-server) data collection. The code to do the actual crawling is written by the user and the tool takes care of all management needed to distribute resources to be collected.
Documentation:
This project is the result of improvements over past efforts made by our research group in collecting data for scientific analysis. Although performance was always a concern during the development of the tool, the main goals were (re)usability, flexibility and extensibility, what explains many of the design and implementation decisions.
The author thanks very much everyone who contributed somehow with ideias, suggestions, observations and corrections. New contributions are still welcome and will be much appreciated.