Auto-play did exactly what it was designed to do, and that’s the problem.
Spotify’s recommendation system is under scrutiny again after a user said autoplay queued a song with racially offensive lyrics following a 1950s holiday playlist. The track closely mimicked old Christmas recordings and appeared to be AI-generated, yet it carried no label or warning.
While Spotify took the song down, the case has renewed concerns about algorithmic moderation and oversight.
AI Song Surfaces Through Auto-Recommendations
A Spotify user first shared the experience on multiple platforms. He’s been listening to a 1950s holiday playlist when auto-play queued up a racist, AI-generated track. The song had that old crooner sound, close enough to vintage holiday music that it didn’t seem out of place.
Then the lyrics hit: “I’m dreaming of a whites only Christmas. Just like the ones I used to know.”
Spotify removed the track after the post spread. However, the “artist” account is still on the platform, with a catalog of inflammatory titles and around 26,000 monthly listeners.
Spotify didn’t label the song as AI-made, but listeners pointed to a few tells. For one, it uses an unnaturally clean “vintage crooner” vocal tone. The artist page also uploads a large volume of similarly styled tracks with inflammatory titles. And, the images used are clearly AI-generated.
One Redditor warned people not to search for it on YouTube. They say the comments were some of the most overtly racist they’d seen in years.
This post kept circulating, and people started asking the obvious question. How did Spotify’s system recommend this in the first place?
A Pattern of Issues
Spotify’s content problems aren’t new.
Back in 2020, a BBC investigation found at least 20 songs on the platform with racist, anti-Semitic, and homophobic content. It also noted that finding them “required no specialist skills or effort.”
Since then, other incidents have surfaced.
Part of the problem is how Spotify’s recommendations work. The system uses collaborative filtering. So, if people with similar listening habits played a song, you might get it too. It also scrapes blog posts and online discussions to tag songs with descriptive terms, and analyzes audio for things like rhythm and genre.
None of that involves actually checking what the lyrics say.
That’s why an AI-generated track that sounds like 1950s holiday music can get recommended just because it fits the pattern. The same logic applies to playlist names.
Spotify’s Response and Its Critics
Spotify first responded after the “monkey hip-hop” playlist spread online. On social media, the company said the “unacceptable language has been removed” and that it was “triple-checking for other offensive terms.”
Months later, in late September 2025, Spotify announced new rules against “spam, impersonation, and deception,” including a filter to catch spammers and promised “AI disclosures for music with industry-standard credits.”
Still, the company didn’t ban AI music, arguing that “music has always been shaped by technology.”
Not all platforms agree, though. Deezer, for example, labels AI-detected music and excludes it from recommendations.
For critics, these responses miss the point. The problem isn’t just bad content slipping through; it’s that the platform built a system without enough safeguards in the first place.
Kieran Press-Reynolds made a similar point in a recent Pitchfork column
But some users say they’re done waiting for Spotify to figure it out.