agoda-property-listing-scraper

Description

This repository contains a Python scraper for extracting hotel property listings from Agoda. The scraper leverages the Crawlbase Crawling API to bypass bot protections and handle JavaScript rendering. The extracted data includes hotel names, prices, ratings, and profile URLs, and is saved in a JSON file for further analysis.

➡ Read the full blog here to learn more.

Scraper Overview

Agoda Property Listing Scraper

The agoda_property_listing_scraper.py extracts:

1. Hotel Name
2. Price
3. Rating
4. Hotel Profile URL

The scraper supports infinite scrolling and ensures comprehensive data extraction from Agoda search listings.

Environment Setup

Ensure that Python is installed on your system. Check the version using:

# Use python3 if you're on Linux/macOS
python --version

Install the required dependencies:

pip install crawlbase beautifulsoup4

1. Crawlbase – Handles JavaScript rendering and avoids blocks.
2. BeautifulSoup – Parses and extracts structured data from HTML.

Running the Scraper

1. Get Your Crawlbase Access Token
   • Sign up for Crawlbase here to get an API token.
2. Update the Scraper with Your Token
   • Replace "YOUR_CRAWLBASE_TOKEN" in agoda_property_listing_scraper.py with your Crawlbase API token.
3. Run the Scraper

   python agoda_property_listing_scraper.py

The extracted hotel data will be saved in hotels_data.json.

To-Do List

• Extract additional details like amenities, number of reviews, and hotel location.
• Add support for exporting data to CSV.
• Implement error handling for failed requests.
• Enhance pagination handling for larger datasets.

This scraper is useful for travel analysts, hotel market research, and pricing comparisons across different locations. 🚀