Step 1: Find the URL that you want to scrape
Step 2: Inspecting the Page
Note: In this project, i have used the website called Hacker News
Step 3: Find the data you want to extract
Step 4: Implement your code
Step 5: Run the code and extract the data
Web scraping is an automated method used to extract large amounts of data from websites. The data on the websites are unstructured. Web scraping helps collect these unstructured data and store it in a structured form. There are different ways to scrape websites such as online Services, APIs or writing your own code.
When you run the code for web scraping, a request is sent to the URL that you have mentioned. As a response to the request, the server sends the data and allows you to read the HTML or XML page. The code then, parses the HTML or XML page, finds the data and extracts it.
To extract data using web scraping with python, you need to follow these basic steps:
Find the URL that you want to scrape Inspecting the Page Find the data you want to extract Write the code Run the code and extract the data Store the data in the required format
On running the above repo, you can get the news which has points > 100 from hacker news website.Hopefully you can use this to read the latest trend news.
Copyright ©lucky 2020