Anime Web Scraper

Download images of anime's episode previews from official and news websites.

Introduction

The Anime Web Scraper is a script that downloads images of previews of episodes from official websites. The scraper also detects and downloads character visuals, Blu-ray cover and bonus illustrations for newer anime (mostly from 2020). The program is written in Python 3.

Click here to see the demo on YouTube.

Motivations

The motivations for building the scraper is to download contents quickly for blogging purpose (e.g. posting on Twitter).

Setting Up

Download and install the latest version of Python
When installing Python, make sure to check 'Add Python 3.X to PATH':
Open the Command Prompt (for Windows) or Terminal (for MacOS).
Run the following command to install all the packages needed to run the program:

pip install -r requirements.txt

Running the Program

Using the Command Prompt (Terminal for MacOS), change to the directory to where the file program.py is located.
Run the following command: python program.py
Select the filtering method (1, 2, or 3) to search for anime to be selected by entering the number. Enter '0' to exit.
1. Filter by Keyword
  1. Enter the keyword to find matching anime. To list all anime available, just press 'Enter' without specifying any keyword.
  2. If a match is found, select the anime by choosing the number(s) beside the anime title.
  3. You can specify which anime to select using the following number format:
    1. Input 3 to select the 3rd anime.
    2. Input 2-5 to select the 2nd to the 5th anime (2nd, 3rd, 4th and 5th)
    3. Input 5,8 to select the 5th and 8th anime.
    4. Input 2-5,7,9-11 to select the 2nd, 3rd, 4th, 5th, 7th, 9th, 10th and 11th anime
    5. Input 0 or any number higher than the number of anime listed to exit without selecting any anime.
2. Filter by Season
  1. A list of season available will be shown.
  2. Select the season(s) you want to display the list of anime that airs in the season(s) specified. You can select multiple season(s) using the same number format as mentioned in the Filter by Keyword section.
  3. Once selected, the list of anime will be shown. Select the anime in the same way as described in the Filter by Keyword section.
3. Filter by Keyword and Season
  1. Refer to the above instructions in Filter by Keyword and Filter by Season sections.
The selected anime will be downloaded.
The images will be saved at the folder download.

Other Commands in the Program

Option 4 - Identify the season the anime belongs to
- Similar to Option 1, filter by keyword, then select the anime to see which season it belongs to.
Option 5 - Download from news website
- Select the news website (Aniverse, MocaNews, Natalie, WebNewtype) to download from.
- Upon selecting, enter the Article ID to download the images in the article.
- Examples:
  - Anime Recorder: Article ID is 12345 from https://anime-recorder.com/tvanime/12345
  - Aniverse: Article ID is 12345 from https://aniverse-mag.com/archives/12345
  - MocaNews: Article ID is 2021010101000a_ from https://moca-news.net/article/20210101/2021010101000a_/01/
  - Natalie: Article ID is 12345 from https://natalie.mu/comic/news/414049
  - WebNewtype: Article ID is 12345 from https://webnewtype.com/news/article/12345/
- The images will be saved at the folder download\news\{website}\{article_id}, where
  - {website} is
    - animerecorder for Anime Recorder;
    - aniverse for Aniverse;
    - moca for MocaNews;
    - natalie for Natalie; and
    - wnt for WebNewtype
  - {article_id} is the Article ID.

Websites

Here are some of the websites that are scraped (click to expand):

News Website

New Anime

The premiere date for the anime listed here has not been announced.

Tsuyokute New Saga

Spring 2025 Anime

Winter 2025 Anime

Fall 2024 Anime

Summer 2024 Anime

Spring 2024 Anime

Winter 2024 Anime

Fall 2023 Anime

Summer 2023 Anime

Spring 2023 Anime

Winter 2023 Anime

Fall 2022 Anime

Summer 2022 Anime

Spring 2022 Anime

Winter 2022 Anime

Fall 2021 Anime

Summer 2021 Anime

Spring 2021 Anime

Winter 2021 Anime

Fall 2020 Anime

Summer 2020 Anime

Spring 2020 Anime

Winter 2020 Anime

Fall 2019 Anime

Summer 2019 Anime

Spring 2019 Anime

Winter 2019 Anime

Fall 2018 Anime

Summer 2018 Anime

Spring 2018 Anime

Winter 2018 Anime

Fall 2017 Anime

Summer 2017 Anime

Spring 2017 Anime

Spring 2016 Anime

Gakusen Toshi Asterisk 2nd Season

Fall 2015 Anime

Notes on Season

The anime are grouped according to the season it first premiered. There are four seasons:

Winter (January to March)
Spring (April to June)
Summer (July to September)
Fall (October to December)

Content Viewer

You may browse the media downloaded by the scraper on a web browser.

Setting Up for PHP

Download the latest PHP. Click here to download.
Set the system environment path to where the PHP is downloaded.
Open Command Prompt/Terminal and input php -v. If the PHP version appeared, this means it is installed successfully.

Instructions

Change directory to where index.html is located.
Run php -S localhost:4000 to run a PHP server at port 4000. You can specify other port numbers.
Open a web browser and type localhost:4000 to access the Content Viewer.

Name		Name	Last commit message	Last commit date
Latest commit History 2,576 Commits
.idea		.idea
anime		anime
image_logs		image_logs
images		images
search		search
static		static
DeveloperGuide.md		DeveloperGuide.md
README.md		README.md
aniverse_guess.py		aniverse_guess.py
aniverse_guess_now.py		aniverse_guess_now.py
extract_log.py		extract_log.py
index.html		index.html
log_to_db.py		log_to_db.py
main.py		main.py
mainday.py		mainday.py
migrate.py		migrate.py
program.py		program.py
requirements.txt		requirements.txt
run.py		run.py
scan.py		scan.py
test.py		test.py

tetrix1993/anime-web-scraper

Folders and files

Latest commit

History

Repository files navigation