Spotify Music Recommendation System

Spotify models the user-track interaction matrix (billions of rows × 100M+ columns) using implicit matrix factorization — factoring the sparse matrix into user and track embedding vectors of ~128 dimensions. Similar users map to nearby points in embedding space, and a user's predicted affinity for an unheard track is the dot product of their embeddings. The model is trained on implicit feedback signals — plays, skips, saves, and playlist additions — weighted by engagement strength. This runs as a massive Apache Spark job processing billions of interaction events.

Spotify crawls the web for music blogs, reviews, and playlist descriptions, then applies NLP techniques inspired by Word2Vec to derive track embeddings from textual context. If two tracks frequently appear in similar textual contexts ('chill vibes for studying'), their embeddings converge. Playlist titles are especially valuable: a user-created playlist called 'Sad Rainy Day Songs' provides semantic labels that no listening data alone could capture. This approach solves the cold-start problem for new tracks — even with zero listens, a track mentioned in music blogs gets meaningful embeddings.

For tracks with zero interaction data and no web presence (the extreme cold start), Spotify trains convolutional neural networks directly on raw audio spectrograms. The CNN learns to extract features like tempo, key, energy, instrumentalness, and genre from the audio waveform. These audio embeddings complement collaborative filtering: two tracks that sound similar are embedded nearby even if they've never been co-listened. The audio model uses mel-spectrograms as input and is trained to predict collaborative filtering embeddings as the target, aligning the audio and interaction spaces.

Every Monday, Spotify generates a personalized 30-track Discover Weekly playlist for 100M+ users. The pipeline runs over the weekend: (1) Compute user and track embeddings via collaborative filtering on the latest interaction data, (2) For each user, find candidate tracks via approximate nearest-neighbor search in the embedding space, (3) Filter out tracks the user has already heard, (4) Apply diversity rules to avoid genre monotony, (5) Rank final candidates using a multi-objective model balancing predicted engagement with exploration. The entire pipeline processes petabytes of data using Spark on Google Cloud.

Spotify runs thousands of concurrent A/B experiments on recommendation algorithms. The key tension is exploitation vs exploration: recommending familiar-sounding tracks maximizes short-term engagement, but users who discover new favorites have higher long-term retention. Spotify uses multi-armed bandit approaches to balance this — allocating a fraction of recommendation slots to exploratory tracks from underrepresented genres or new artists. Each experiment tracks both immediate signals (skip rate, save rate) and long-term metrics (30-day retention, subscription conversion) to avoid optimizing for shallow engagement.