Scrapy-Crawlers

Web Scraping with Python using Scrapy, Splash and Selenium.

Installation

• Cloning Repo:

git clone https://github.com/ADI-KOTKAR/Scrapy-Crawlers.git

• Installing Requirements:

pip install requirements.txt

Resources

• CSS : https://try.jsoup.org/

• XPath : https://scrapinghub.github.io/xpath-playground/

• IMDB : https://www.imdb.com/

• Livecoin : https://www.livecoin.net/en

• DuckDuckGo : https://duckduckgo.com/

• worldometers : https://www.worldometers.info/world-population/population-by-country/

• slickdeals : https://slickdeals.net/computer-deals/

Running Splash on Docker

docker run -it -p 8050:8050 --rm scrapinghub/splash