Skip to content

Latest commit

 

History

History
26 lines (16 loc) · 758 Bytes

README.md

File metadata and controls

26 lines (16 loc) · 758 Bytes
Raw

README

This README would normally document whatever steps are necessary to get your application up and running.

What is this repository for?

  • craw data from ptt(web based)
  • save data into mysql DB

How do I get start?

  • install composer
  • make init
  • make
  • ./crawler --help

Cron job example

0 0 * * * /path/to/crawler --board=politics --sleep-between-retry=10 --start-date=$(date +\%Y-\%m-\%d --date='-3 day') --storage=rdb --db-username=<username> --db-password=<password> --db-host=<host ip> --db-port=<port> --timeout=10 --sleep-between-article=1 --stop-date=2014-01-01 --stop-on-duplicate=true > /tmp/politics.log 2>&1

LICENSE

GPLv3