Skip to content

Commit

Permalink
v2.7.0
Browse files Browse the repository at this point in the history
- Added --workers to adjust pool thread workers for Thread Pool Scraper
- Use Firefox instead of Chrome
- Adjusted wait time for clicking pop-up ad to 2 seconds
- Adjusted scroll-down length from 2,000 to 8,000 pixels
- Adjusted logger from loguru to use Standard Output instead of Standard Error for logging that printed in terminal
  • Loading branch information
sakan811 committed Jun 12, 2024
1 parent b56c1eb commit ee76a03
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion japan_avg_hotel_price_finder/scrape.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ def scroll_down_until_page_bottom(driver: WebDriver) -> None:
logger.debug(f'{current_height = }')

# Scroll down to the bottom
driver.execute_script("window.scrollBy(0, 4000);")
driver.execute_script("window.scrollBy(0, 8000);")

# Get current height
new_height = driver.execute_script("return window.scrollY")
Expand Down

0 comments on commit ee76a03

Please sign in to comment.