Automatic thumbnail generating program for Youtube
for ybigta 2020-2 Conference
- Individual streamers want to upload their streaming video to Youtube
- However, it is difficult to make Youtube thumbnail for the streamers who don't have their own broadcast editor
- THEN, how about a program which can make high-quality thumbnail AUTOMATICALLY?
- Getting comments and preprocessing
- Highlight extraction
- Selecting thumbnail image
- Selecting thumbnail typography
- Making thumbnail
-
Getting comments
- Used Twitch API to get comments of twitch videos
- Got time, nickname, and chatting contents from comments
-
Preprocssing
- Committed feature engineering for highlight detection
- Highlight detection
- Utilitized anomaly detection to get highlight area from video
- Isolation Forest was the best model for the task
- Removed anomalies which are not hightlights. Ex) start of the video
- Extracting highlights
- Used twitch licker software to get specific period of video
Select thumbnail image as the scene which streamer showing extreme emotions
-
Face detection
- Used Haar Cascade Classifier to detect streamers' face in a video
-
Emotion classifcation
- Committed CNN-based emotion classification
- Seleceted the moment streamers' emotion percentages are extremely high
-
Keyword extraction
- Extracted keyword frequently commented from comments
- Used Soynlp package to apply gramatical whitespaces to the comments
-
Object seperation
- Used sematic segmentation to seperate people and background
- Combined new background based on extracted keyword
-
Combine typography with thumbnail
