GitHub - runmycode/rmcscrape: scripts and scrapes of files from the legacy runmycode site

runmycode / rmcscrape Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

scripts and scrapes of files from the legacy runmycode site

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
site_scrapes		site_scrapes
.gitignore		.gitignore
README.txt		README.txt
extractcontent.py		extractcontent.py

Repository files navigation

We don't have access to the legacy databases or filesystems. This
directory contains a script for scraping CompanionSite pages that were
obtained from a wget command.

Directory Structure
-------------------

site_scrapes/
    contains site.do files copied from www.runmycode.org/CompanionSite/site.do?siteId=*
    from running this wget:
    wget -m -olog --random-wait --save-headers http://www.runmycode.org

extractcontent.py
    a python script that runs over the site.do files and extracts
    resulting site metadata in to a file called results.json