INTRO:
Our project “Moodify for Spotify” aims to survey a handful of user-generated playlists from Spotify and use these playlists’ titles to associate individual songs with sentiments or feelings. A preliminary goal is to categorize each of these songs within a specific “mood” category from a set of predefined moods or feelings. The way we are planning to construct this set is by either:
(1) hardcode a wide set of feelings that we think are extensive enough to cover our dataset, or
(2) use the playlists’ titles as a starting point to generate this set depending on the feelings depicted in the dataset.
We are then planning to associate each feeling with a set of related words that depict that feeling and use that to bind each song to its corresponding category/ies. Our ultimate goal, is to use these sets of songs to generate more accurate playlists (by reshuffling the songs in the original playlists) that contain songs reflecting the feelings associated with their titles.
Our data:
Our data was provided by Spotify as a part of their 2018 RecSys Challenge. The current data we have is a 4000 playlist subset of the ‘Million Playlist Dataset’ available during the Challenge obtained from Github. We emailed Spotify to ask for the actual dataset, which would allow us to generate more accurate insights.
The data is stored as a list of JSON objects: each playlist is a a JSON object, and in each playlist object, there is a “track” key whose value is a list of JSON objects, with each of those JSON objects representing a song.
The data was supposed to be already cleaned, but we checked over it to make sure that it was clean, and it is indeed clean.
Next Steps:
Data Visualization:
We want to have a rough draft of visualization. We are considering using the following libraries:
- D3
- Matplotlib
- Seaborn
D3 is a JavaScript library, while Matplotlib and seaborn are Python libraries. As such, we are leaning towards Matplotlib and seaborn because we are more comfortable with Python as a language. We also want to figure out what kinds of charts and tables we want to use that would communicate our data best to the user.
ML Clustering:
We are planning to use some of the ML clustering techniques we learn in class to extract a set of feelings that spans our dataset from the playlist titles. We are then planning to use similar techniques, alongside sentiment-analysis and possibly NLP, to examine and map each song to the set(s) of feelings that it depicts. We are still not entirely sure how are we going to do that but we are waiting to see how deep we go into ML clustering to decide exactly on the methods that we will use. We are also planning to use the hypothesis testing procedure we learned in class to test different ML models and decide on the best-fit algorithm for our project.
Emotion Metric:
We also want to figure out a metric that we can give to songs for their association with a particular mood/activity.
There are two possible methods we’re thinking of categorizing the playlists. Firstly, we can determine the list of categories, then fit the playlist names into each category. Alternatively, we can look at the data itself and the existing playlist names and bucket them accordingly.
Going deeper into exploring these, we wanted to gain inspiration for lists of emotions, and found the following three.
- Robert Plutchick’s
- Fear → feeling of being afraid, frightened, scared.
- Anger → feeling angry. A stronger word for anger is rage
- Sadness → feeling sad. Other words are sorrow, grief (a stronger feeling, for example when someone has died)
- Joy → feeling happy. Other words are happiness, gladness
- Disgust → feeling something is wrong or nasty.
- Surprise → being unprepared for something.
- Trust → a positive emotion; admiration is stronger; acceptance is weaker.
- Anticipation → in the sense of looking forward positively to something which is going to happen. Expectation is more neutral.
- Book Two of Aristotle’s “Rhetoric”
- Anger, opposite calmness (not feeling excited)
- Friendship, is where people have a bond of joy and will come together and have fun
- Fear, opposite courage (having courage in the face of fear)
- Shame, opposite confidence (shame: how one feels about one’s past bad actions or thoughts; shamelessness: one does not feel shame, but others think one should)
- Kindness (benevolence), opposite cruelty (kindness: when people are good to other people)
- Pity (when people feel sorry for other people)
- Indignation (feeling angry because something is not fair, such as undeserved bad fortune)
- Envy, jealous (pain when people have something that one wishes for oneself)
- Love, a strong emotion of attachment one feels for someone else. Ranges to family, pets, friends, significant others or fictional characters.
- Darwin’s ideas (The Expressions of the Emotions in Man and Animals)
- Suffering and weeping
- Low spirits, anxiety, grief, dejection, despair
- Joy, high spirits, love, tender feelings, devotion
- Reflection, meditation, ill-temper, sulkiness, determination
- Hatred and anger
- Disdain, contempt, disgust, guilt, pride, helplessness, patience, affirmation and negation
- Surprise, astonishment, fear, horror
- Self-attention, shame, shyness, modesty, blushing.
Drawing from those 3 possible lists, the categories for playlists we can imagine looking for are:
- Anger (rage, frustration)
- Sadness (reflection, meditation, sulkiness, suffering, weeping, life sucks),
- Joy (high spirits, happy, optimistic),
- Love (romantic),
- Surprise (excitement, anticipation),
Other than emotions, we can also group by categories of activities of playlists such as:
- Gym,
- driving,
- studying,
- sleep,
- sex,
- shower,
- Parties,
- outdoors
The other thing we can do is try to look at the existing playlist names, and see if we can bucket them/categorize them based on what we already see, but how do we do that? Possibly through Sentiment Analysis
Sentiment analysis:
Here is what we’ve learned about sentiment analysis, through the website: https://monkeylearn.com/sentiment-analysis/
- What is sentiment analysis?
- A field in Natural Language Processing (NLP) that builds systems to attempt to identify and extract opinions from text. Aside from extracting opinions, sentiment analysis also attempts to identify exact attributes from the text: 1) polarity (positive or negative opinion), 2) Subject, 3) Opinion holder.
- Opinions are subjective expressions describing peoples sentiments, appraisals, feelings on a particular subject, whereas facts as objective. There are two types of opinions
- Direct opinions
- “The picture quality of computer screen A is poor”
- Comparative opinions
- “The picture quality of computer screen B is better than screen A”
- Direct opinions
- Sentiment analysis is a classification problem with two subproblems
- Subjectivity classification
- Polarity classification
- Scope of sentiment analysis can be by
- Document
- Sentence
- Or sub sentence level
- Types of sentiment analysis
- Fine-grained (5 categories from very negative to very positive)
- Emotion detection
- Emotion detection resort to lexicons or complex ML algorithms
- Aspect-based sentiment analysis
- Intent analysis
- Sentiment analysis approached
- Rule based
- Perform analysis based on set of manually crafted rules
- Automatic
- Rely on machine learning
- Hybrid
- Rule based
- Automatic analysis
- Machine learning classifier can be implemented with these steps and components:
Besides parsing words for specific feelings, many playlist titles contain emojis and symbols. We hope to isolate these emojis and perhaps map them to keywords using Twitter’s emoji library to be used in our general sentiment analysis. We can do this through considering “emoji lexicons”, databases of emojis alongside what the emoji represents. Examples of these can be seen here:
CS1951a: Data Science 