Amazon-Scraper

Usage

Amazon is an American Multinational Corporation focusing on a variety of industries, like e-commerce, cloud computing, online advertising, and artificial intelligence. We, in this project focus on the e-commerce platform hosted by amazon, on the web-app Amazon.in The functionalities of each runnable file is given in the following sections of this file.

Pre-Requisites

• Python version 2.7 and above
• Selenium module installed
• CSV module installed
• Time module installed

To install the above mentioned libraries using pip, input the following commands in the terminal

pip install selenium

Note: CSV and Time module comes pre-installed with Python's standard utility module

File Details

• Main.py

  • Pre-defined: URL: Amazon Bags Search
  • Function: Traverses to the above URL. Extracts the following data
    • URL
    • Name
    • Price
    • Ratings
    • Number of Ratings
  • Gathers the above mentioned information for upto 10 pages (Hard coded, can be changed)
  • Output: CSV files for each page.
  • Features :
    • Custom user agent to mimic real browser
    • Custom scrolling, to overcome dynamic page load
    • Explicit wait to reduce bot suspicion
    • Explicit wait before writing to csv file

• Part2.py

  : Under construction!!

Note: Please find the privacy policy in Privacy.MD

Any tips for improvements or comments are welcomed