opencrawler.1

.TH OpenCrawler 1

.SH NAME OpenCrawler v 1.0.0

.SH DESCRIPTION

.B OpenCrawler v 1.0.0
Is an program for crawling through websites.

.TP
An open source crawler/spider. LICENSE - MIT.

.SH COMMANDS

.TP
.BR help
Get info about the commands

.TP
.BR v
Get the version of open crawler

.TP
.BR crawl
Starts up the normal crawler

.TP
.BR forced_crawl <website>
Forcefully crawl a website/Make the crawler crawl the website

.TP
.BR crawled_status
Shows the amount of data in DB , etc

.TP
.BR configure
Write / reWrite The config file

.TP
.BR connection-tree <website> <no of layers> 
Makes a tree of websites connected to it, layers by default is 2

.TP
.BR check_html <website>
Checks if a website respond with html content

.TP
.BR crawlable <website>
Checks if a website is allowed to be crawled

.TP
.BR dissallowed <website>
Lists the websites not allowed to be crawled

.TP
.BR re-install
Re installs the Open Crawler

.TP
.BR update
Updates the open crawler

.TP
.BR install-requirements
Installs requirements for open crawler

.TP
.BR search <search>
Search from the crawled data


.TP
.BR fix_db
Tools to fix the DB