Skip to content
This repository has been archived by the owner on Jul 31, 2024. It is now read-only.

A example of a git scraper that download, lints, commits and archives a data set

License

Notifications You must be signed in to change notification settings

palewire/git-scraper-example

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

72 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

git-scraper-example

A example of a git scraper that downloads, commits and archives a data set.

The source for this demonstration is the list of liquor stores in the state of Iowa. The scraping routine is automated by the scrape.yaml workflow.

Instructions

Clone the repository

git clone git@github.com:palewire/git-scraper-example.git

Move into the directory

cd git-scraper-example

Install the Python tools

pipenv install

Run the downloader

pipenv run python src/download.py

About

A example of a git scraper that download, lints, commits and archives a data set

Topics

Resources

License

Stars

Watchers

Forks

Languages