README.txt

# Crawlyn
#########

Experimental crawler to grab data from websites, including:

* Internal links.
* Plain-text and obfuscated email addresses.
* Plain-text and obfuscated phone numbers. Supported formats:
  - 0123456789, 01 23 45 67 89
  - +33 1 23 45 67 89
  - 3334445555, 333 444 5555, 333 4445555
  - 333.444.5555
  - 333-444-5555, 333444-5555
  - (333) 444 5555, (333)4445555

This project is for educational purposes only.

## Installation

pip install git+https://github.com/iamcryptoki/crawlyn.git

## Usage

Basic usage:

$ crawlyn https://www.example.com/

$ crawlyn https://www.example.com/ https://fakedomain.org/

The results are printed to the console in JSON format.

Save results to a file:

$ crawlyn https://www.example.com/ --output /path/to/results.json

Running Crawlyn with a proxy:

$ crawlyn https://www.example.com/ --proxyhost 127.0.0.1 --proxyport 9999 --proxytype socks5

Both proxy host address and port number needs to be set.

Default proxy type: http.

Running Crawlyn with Tor:

$ crawlyn https://www.example.com/ --tor

## License

This code is released under a free software license and you are welcome to fork it.