It can feel like these apps are reading minds. They aren't — they're running a machine learning model on patterns in behaviour, at a scale no individual person could track by hand.
The machine learning article in this series described a model that learns patterns from data rather than following explicitly programmed rules. A recommendation system is one of the most common everyday applications of exactly that idea: a model trained on huge amounts of behaviour data — what people watched, listened to, or bought, and what they did afterward — learning patterns well enough to predict what a specific person is likely to want next. This guide covers the two main approaches behind that prediction, and where they still fall short.
At its core, a recommendation system is trying to answer one question for every item in a huge catalogue: how likely is this specific person to enjoy this specific thing? It does this by learning from data about what similar people have liked, and about what this person has liked before, rather than by understanding taste the way a person would explain it in words.
A recommendation isn't a genuine understanding of taste — it's a predicted number, roughly "how many stars would this person likely give this item," ranked from highest predicted score to lowest and shown as a list.
Collaborative filtering is the older and more widely used of the two main approaches. It works by finding patterns across many users' behaviour, without needing to know anything about the actual content of the items themselves. If two people have rated or watched many of the same things similarly in the past, the system predicts that whatever one of them liked but the other hasn't seen yet, the other person is likely to enjoy too.
| Approach | What It Uses |
|---|---|
| User-based | Finds other users with similar past behaviour and recommends what they liked |
| Item-based | Finds items that tend to be liked by the same people and recommends similar items |
Notably, collaborative filtering never needs to understand what a movie is actually about — it only needs the pattern of who liked what, which is why it can work equally well for movies, songs, or products with no changes to the underlying method.
Content-based filtering takes the opposite approach: it looks at the actual characteristics of items a person has liked before — a song's tempo and genre, a movie's actors and category — and recommends other items with similar characteristics. Unlike collaborative filtering, this approach doesn't need other users' data at all; it can make a reasonable recommendation for a brand-new user based purely on the few items they've already interacted with.
Most large streaming and shopping platforms combine both approaches, called a hybrid system — using content-based filtering to handle brand-new users or newly added items with no behaviour history yet, and collaborative filtering once enough behaviour data exists, to capture patterns that content features alone would miss.
Collaborative filtering has one clear weakness: it needs behaviour data to find patterns from, so it struggles with anything genuinely new — a brand-new user with no history, or a newly released item nobody has interacted with yet. This is called the cold start problem, and it's the specific reason most real systems blend in content-based filtering, which can make a reasonable guess from an item's characteristics alone, even with zero behaviour data to learn from.
Because a recommendation system is optimised to show more of what someone is statistically likely to engage with, it can narrow what a person sees over time — repeatedly surfacing similar content and rarely surfacing anything that doesn't match established patterns. This effect, sometimes called a filter bubble, is a genuine and actively debated trade-off of highly personalised recommendations: the same optimisation that makes a feed feel relevant can also make it feel repetitive or narrow, and it's a topic connected to the bias concerns the AI ethics article in this series covered.
A recommendation system isn't trained once and left alone — it continuously updates based on new behaviour. Every click, skip, watch, or purchase becomes new training data, refining future predictions. This means the system is always learning from a mix of genuine preference and its own past recommendations, since people can only interact with what they were shown in the first place, which is part of why filter bubbles can reinforce themselves over time.
A recommendation feed that feels eerily accurate isn't reading anyone's mind — it's running the same pattern-learning process covered throughout this series' machine learning articles, applied to an enormous, constantly updated set of behaviour data. That's genuinely useful for finding something worth watching or listening to, but it's worth remembering that a system optimised to predict what's already liked isn't the same as a system built to broaden what gets discovered.
10 questions. Select an answer for each, then submit to see your score instantly.