RomanEmperorsScraper is a product of the collaboration of Radu Ionut Barbalata, Matteo Collica, Ivan Dalla Ragione and Sara Lazzaroni as a university project for the oral examination of the "Metodologie di Programmazione" teaching at Sapienza University of Rome.
The original requirements concerned the creation of one or more family trees relating to each dynasty of Roman Emperors, with the aim of visualising the various parental relationships that exist, through the technique of web scraping and the use of the Selenium framework.
Our group subsequently implemented many additional functionalities, such as the possibility of importing and exporting data obtained through scraping and an interface for visualising the information, also obtained through scraping, of the dynasty members. Furthermore, in contrast to some other works on the subject, we have preferred an approach that allows the user to choose the dynasties he or she wants to scrape, rather than starting, and thus making the user wait for, the scraping of all dynasties when starting the program. Particular attention was also paid to maintaining the possibility of using the program with any browser among those supported by Selenium and with any operating system based on Windows, macOS or Unix.
The project was finally presented orally on 15 July 2022 to Professors Gabriele Etta and Walter Quattrociocchi, receiving a grade of 30 cum laude.
Thanks to the use of Maven and the corresponding configuration file (pom.xml) distributed in this GitHub repository, you can import the project into your favourite IDE (in our case it was Intellij IDEA, but we also tested it on Eclipse for example) and easily compile and run it.
-
An operating system Windows, macOS or Unix-based;
-
Firefox, Chrome, Edge, Internet Explorer or Safari - having at least one of them installed is required by Selenium (and therefore by our program) to be used for scraping.
To facilitate the running of the program we packaged it as a JAR for you, so that running it is as easy as double clicking on the JAR file or executing the following command on your terminal:
java -jar RomanEmperorsScraper.jar
Some additional options are also provided when executing from a terminal:
- Add a -t argument to the command to run some tests of the program's correct functioning instead of the program itself;
- Add a -d argument to print some additional debug information during the program's execution.
Please ensure to execute the shipped JAR with a Java version later than 8 (as it is the one we, developers, compiled and packaged as JAR the project with).
If you wish to know more about our classes and methods please feel free to check our documentation comments in source files or the Javadoc itself.
Some other icons used in our program (like those named with a '_roll' suffix) are modified versions of the enumerated ones.
- Source: Freepik (Author: rawpixel-com)
- Source: Clipground