Skip to content

Latest commit

 

History

History
176 lines (147 loc) · 4.11 KB

README.md

File metadata and controls

176 lines (147 loc) · 4.11 KB
Raw

priceCrawler

This is a spring boot service. The service has various APIs that can help crawl Amazon(.com) and extract the SKU details. It uses PostgreSQL as database for storing data. The code uses Strategy Pattern inorder to support adding algorithms for new retailer like Walmart, Instacart etc.

How to run?

  • Database used is PostgreSql. Install it in your machine.
  • Create a database commerce with user postgres and password postgres
  • Alternatively, you can change the configuration in application.properties
  • In the terminal run mvn spring-boot:run.
  • The server will be up and running.

APIs

GetHtml

/products/html - GET

Query Params -

  • sku = string
  • url = string
{
   "timeTaken": "81 ms",
   "data": {html}
}

Example - http://localhost:7007/products/html?sku=B07Q7L6SRG

GetProductDetails

/products - GET

Query Params -

  • sku = string
  • url = string
	Response
{
   "timeTaken": "81 ms",
   "data": {
       "id": "cd8ef413-6203-4dd7-b537-8bbcd0eeed7f",
       "sku": "B07Q7L6SRG",
       "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
       "description": "Mom's…..",
       "offerPrice": 72.0,
       "createdAt": "2021-05-09T22:22:33.803+00:00",
       "ratings": {
           "3star": "9%",
           "2star": "2%",
           "4star": "25%",
           "overallCount": "12,945",
           "5star": "60%",
           "1star": "3%"
       }
   }
}

Example - http://localhost:7007/products?sku=B07Q7L6SRG

GetProductDetailsHistory

/products - GET

Query Params -

  • sku = string
  • url = string
  • date=dd/MM/yyyy Note - you need to pass date
	Response -
	{
   "id": "a5d62e3b-9c77-41bb-9cef-149ed816043f",
   "sku": "B07XMLWH7J",
   "title": "Kellogg's Corn Flakes Original, High in Iron, High in B Group Vitamins, Breakfast Cereals, 1.2 kg Pack",
   "description": "Don't compromise on morning nourishment when you have a balanced, great-tasting breakfast that is high in iron, vitamin b1, b2, b3, b6 and vitamin c, it is quick to eat and gives you energy even after 3 hours.",
   "offerPrice": 298.0,
   "createdAt": "2021-05-09T15:56:26.758+00:00",
   "ratings": {
           "3star": "9%",
           "2star": "2%",
           "4star": "25%",
           "overallCount": "12,945",
           "5star": "60%",
           "1star": "3%"
       }
}

Example - http://localhost:7007/products?sku=B07XMLWH7J&date=10/05/2021

List All crawled Products and details till now.

/products - GET

Response
{
   "timeTaken": "275 ms",
   "data": [
       {
           "id": "cd8ef413-6203-4dd7-b537-8bbcd0eeed7f",
           "sku": "B07Q7L6SRG",
           "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
           "description": "Mom's....",
           "offerPrice": 72.0,
           "createdAt": "2021-05-09T22:22:33.803+00:00",
           "ratings": {
               "3star": "9%",
               "2star": "2%",
               "4star": "25%",
               "overallCount": "12,945",
               "5star": "60%",
               "1star": "3%"
           }
       },
       {
           "id": "0b7d8f2f-d0b9-4573-a84d-1322b8a281de",
           "sku": "B07Q7L6SRG",
           "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
           "description": "Mom's....",
           "offerPrice": 72.0,
           "createdAt": "2021-05-09T22:57:30.968+00:00",
           "ratings": {
               "3star": "9%",
               "2star": "2%",
               "4star": "25%",
               "overallCount": "12,945",
               "5star": "60%",
               "1star": "3%"
           }
       }
   ]
}

Example - http://localhost:7007/products

PriceTrend

/products/trends

Response 
{
   "timeTaken": "85 ms",
   "data": [
       {
           "sku": "B07Q7L6SRG",
           "timestampPrices": [
               {
                   "timestamp": "2021-05-09T22:57:30.968+00:00",
                   "price": 72.0
               },
               {
                   "timestamp": "2021-05-09T22:22:33.803+00:00",
                   "price": 72.0
               }
           ]
       }
   ]
}

Example - http://localhost:7007/products/trends