What makes a popular song popular? The beat, the lyrics, the artist’s name? Hit Song Science, a branch of music information retrieval, works from the assumption that hit songs share measurable features, and tries to predict whether a track will chart before it is released. For an unsigned artist, a producer, or a label, knowing that in advance saves money and time. Kai Middlebrook and Kian Sheik took on the problem in a 2019 paper, using Spotify data and four different models.
Building the dataset
The authors first pulled a large set of tracks and their audio features through the Spotify API. They then collected every unique song that had appeared on the Billboard Hot 100 through the Billboard API, restricting both sets to tracks released between 1985 and 2018. Matching the two on title and artist gave a combined dataset of roughly 1.8 million songs, of which about 12,000 had been Hot 100 hits.
That imbalance, roughly one hit per 150 songs, would let a model score well by predicting “not a hit” every time. So they built a third, balanced dataset of about 12,000 hits and 12,000 non-hits for training. Each track carried 27 features in four groups: track information, artist information, album information, and audio analysis features such as tempo, energy, and danceability. After processing, the data was split into training, validation, and test sets.
The models
Four classifiers were trained to predict Billboard success: logistic regression, a neural network, a random forest, and a support vector machine (SVM).
Results
The random forest and the SVM outperformed the other two on accuracy. The random forest was the most balanced model, with test accuracy of 0.877, precision of 0.86, and recall of 0.90. Logistic regression and the neural network sat around 0.82 accuracy.
The SVM is the interesting case. Its accuracy was 0.839, but its precision was 0.995, meaning that when it called a song a hit it was almost always right. The cost was recall of 0.704: it missed nearly 30 percent of the actual hits. The authors point out that a label may prefer exactly that trade-off, since a model that rarely predicts a false hit is one you can act on with less risk.
What comes next
Hit prediction gives anyone in the industry a way to decide where to spend: an artist might put money into the tracks the model favours rather than into promotion for all of them. The authors suggest that richer inputs, such as lyrics, genre, and release year, could push the predictions further.