Automate a browser to scrape multiple sites to extract data and images (Beautifulsoup, Splinter). Once extracted, storing the info in a NoSQL database (MongoDB). Then using Flask, create a web application to display the data and images.
- Pandas
- Beautifulsoup
- Splinter
- PyMongo <br>
- Scraping.py
- Beautifulsoup and Splinter; drill down into the HTML tags to extract the most recent news article and summary from Mars News site
- Beautifulsoup and Splinter; drill down into the HTML tags to extract the most recent image from Space Images site
- Pandas; scrape Mars/Earth table and read as DataFrame then convert into HTML from Mars Facts site
- Beautifulsoup and Splinter; drill down into the HTML tags to extract all four full size hemisphere images from Mars Hemisphere site
- Store scraped data in MongoDB database
- App.py
- create app to connect MongoDB through Flask to set up web page
- Index.html
- create containers to display news article, images, and table
- create 'Scrape New Data' button to perform our scraping function
- refactor code to ensure data displayed is responsive to multiple device sizes
- Bootstrap; customize web page appearance(background color change, jumbotron image fill, button color change, text color change)
Our Flask app displays the most recent article scraped from the NASA Mars News site, an updated image from the Space Images site and a table from the Mars Facts site. Four hemisphere images scraped from the Mars Hemisphere site are displayed at the bottom. The page is responsive to desktops, tablets, and phones.
E-mail: boyerjason700@gmail.com