Reading Taylor Swift's albums with natural language processing
- Article
- Music
As Taylor Swift released re-recorded versions of her early albums, a Codecademy article by Ian Freed offered a different way of revisiting them. Freed used natural language processing to look at what her songs are about and how those subjects changed as she moved from teenage country singer to pop star.
The data and the method
The starting point was a Kaggle dataset containing every lyric from Swift’s 2006 self-titled debut through Reputation in 2017, one row per line. Using pandas, Freed aggregated the lines up to the level of whole songs. He then fit a topic model to the lyrics. A topic model finds sets of words that tend to appear together and groups them into topics, without being told in advance what the topics should be. Each song is then described by how much of each topic it contains.
Six topics came out of the model, which Freed labelled from their most frequent words: love and beauty, growing up, home, bad or remorse, contemplative, and dancing. To see how they changed over time, he counted the number of songs per topic on each album and plotted the counts by year.
What the timeline shows
Freed drew several inferences from the chart:
- Dance topics rise over the period, matching Swift’s move from country to pop.
- Growing up is most prominent around 2010, the year she moved from Nashville to New York.
- Love and beauty and bad or remorse trade places throughout her career, one rising as the other falls.
- Red (2012) has the most songs about love and beauty of any album.
- Bad or remorse spikes in 2014, the period of her public dispute with Kanye West.
- Contemplative topics fall away by 2017, which Freed reads as greater confidence.
He also noted where the model fails. Reputation, from a period he describes as good for Swift personally, does not show the corresponding rise in love and acceptance topics. A topic model only sees words, and a happy song and a bitter one can share a lot of vocabulary.
Why it is worth trying
The appeal of this project is how little it needs: a public dataset, pandas, and a topic model. From those, it produced a readable narrative across a decade of songwriting, along with an honest note about where the narrative and the data disagree. The same approach works on any body of text with a timeline, from speeches to product reviews. The natural follow-up question the post asked was what corpus readers would want to apply it to.
Source: The Machine Learning Process: Look What Taylor Swift Made Me Do, Ian Freed, Codecademy