Music Recommendation by Facial Emotion Detection using FER, SpotifyAPI, Flask, OpenCV2

Inside-Out

Inside-Out is a web application that uses FER library, OpenCv's Haarcascades to detect faces and their emotion. Music is recommended using Spotify API. The website uses Python, Flask, SocketIO, HTML, CSS & JS.

Inside-Out detects 7 emotions -

• Neutral
• Happy
• Sad
• Disgust
• Fear
• Angry
• Surprise

Video Demo

Screenshots

Setup

• Clone the repository

git clone https://github.com/priyanka-maz/music-recommendation-facial-emotion.git

• Install requirements

cd music-recommendation-facial-emotion
pip install -r requirements.txt

• Go to https://developer.spotify.com/dashboard > Create app (fill the details and submit) > Settings

• Copy ClientID and ClientSecret, paste it in .env

• Run app.py

Use

• Open the webpage.
• Allow camera access permissions.
• Make sure you are in a well lit environment.
• The background color of the page changes depending on the mood.
• Snap your funny expression.
• Wait for the recommended music to pop up.

Make sure you are signed in to Spotify to get the entire track.

Spotify API

1. Once the emotion is detected, the Spotify API /recommendations endpoint is used to obtain the recommended tracks.

2. Some of the parameters the endpoint takes are seed_genres, seed_artists, danceability, energy, loudness, acousticness, valence, tempo etc.

3. Specific emotions corelate to specific parameter values.

4. The parameter values for each emotion was determined by feeding tracks from preexisting playlists (of that particular emotion type) into another Spotify API endpoint /audio-features

5. The parameter values evaluation is performed in spotifyplaylists.ipynb

Note

Make sure you change the client_id and client_secret before running the notebook.

Learn more about Spotify API endpoints Spotify Web API

File Structure

.
├── LICENSE
├── README.md
├── __pycache__
│   └── spotifyaccess.cpython-39.pyc
├── app.py
├── genre names.txt
├── requirements.txt
├── screenshots
│   ├── About.png
│   ├── Emotion Angry.png
│   ├── Emotion Disgust.png
│   ├── Emotion Fear.png
│   ├── Emotion Happy.png
│   ├── Emotion Neutral.png
│   ├── Emotion Sad.png
│   ├── Emotion Surprise.png
│   └── FER dataset viz.png
├── spotifyaccess.py
├── spotifyplaylist.ipynb
├── static
│   ├── connection.js
│   ├── emotion_thumbnail.png
│   ├── fav.png
│   ├── haarcascade_frontalface_default.xml
│   ├── inside_out.png
│   ├── moodmusic.png
│   ├── moodmusic_white.png
│   ├── opencv.js
│   └── style.css
└── templates
    └── index.html