Why Your Recommender Algorithms Can Feel Eerily Spot-On

Why Your Recommender Algorithms Can Feel Eerily Spot-On

You know the unsettling feeling you get when an ad for a super-specific item you searched for maybe once just mysteriously appears on your Instagram feed? You might ask yourself, How did they know?! Well, the answer is recommender systems, algorithms that use data about products and users’ preferences to make recommendations for the best options to choose.

Recommender systems make up a uniquely challenging (but exciting!) area of artificial intelligence. Whereas a predictive machine learning algorithm is designed to come up with one correct answer, recommender system algorithms are preference-based. As a developer, you need to carefully consider human emotion, behavior, ethics, and logic when building a recommender system.

The thing is, recommender systems are not meant to freak out users — quite the opposite, actually. Recommender systems use a handful of techniques to capture users’ interests and tastes, and help them make decisions about everything from what to buy to who to date.

In our new path Build a Recommender System you’ll learn how to use Python and machine learning to create a recommender system. You’ll also get to know the differences between various recommender system techniques, and will understand how to measure the success of a recommender system. Read on to learn more about the elements that make a recommender system good — and sometimes too good.

Factors that make a “good” recommender system

A recommender system performs well if it strikes the right balance between randomness and specificity — if a recommender system suggests something that’s too specific, it can feel off-putting to a user. When you’re building a recommender system or imagining how you might implement one, you have to keep these  factors in mind.

Relevance: This might sound obvious, but it’s important that recommender systems make recommendations that are relevant to the user, meaning it’s very likely that a user will like what’s put in front of them. For example, a dog owner who only buys dog food or browses dog toys on an online pet retailer shouldn’t get recommendations for, say, bird seed or fish tanks.  

Novelty: On the other hand, recommender systems should present recommendations that a user hasn’t seen before so they can discover more items that are relevant to their interests. Going back to the dog owner example, if they’re only getting recommendations for the most purchased or highest-rated dog foods, the user might not be satisfied with their experience shopping with the retailer.

Serendipity: People are typically delighted when recommender systems make recommendations that are unexpected but relevant. The dog owner in our scenario might get a recommendation for grooming tools or dog clothing, items that they haven’t shopped for, but would presumably like given their interests and previous shopping activity.

Diversity: Having a diverse array of items to offer users increases the chances that the user will like at least one of them. Back to the pet retailer: If a pet owner always buys one brand of dog food from a site, but constantly gets recommended more dog kibble, that could be annoying or redundant. A recommender system that prompts other items — in this case, dog supplies like bowls and beds — could motivate a dog owner to add more things to their cart.

Technical Complexity: Given how ubiquitous recommender systems are, organizations need engineers who can understand and interpret the complex algorithms and maintain the parts as needed. You can learn the technical skills that go into creating and maintaining a recommender system with our path Build a Recommender System. If you’re new to coding, don’t sweat it — we’ll teach you Python and machine learning basics in this Codecademy path.

Why recommender systems can seem almost too good

Even though recommender systems are all around us, their effectiveness or accuracy can occasionally catch users off-guard. As a user, you might not realize how much data is being captured when you’re online shopping or using an app.

The way that developers can measure a user’s preferences is by examining how they rate items. Sometimes ratings are very explicit, like giving a book 5 stars on your Goodreads app. But even subtle behaviors — like how many times we view a specific pair of shoes or how long we spend on a page — can be interpreted as an implicit endorsement of the item.

The challenging thing is that recommender systems aren’t an exact science, because our preferences, interests, and internet behaviors may change over time, Nitya says. There’s a “mutually reinforcing feedback loop” between a user and the machine that creates a constant puzzle for developers to tackle, she says.

In the path Build a Recommender System, we’ll demystify the different types of data that’s used in a recommender system, and teach you to create a mathematical model that determines a user’s preferences.

The bottom line

“There’s no such thing as a ‘good’ algorithm or a ‘bad’ algorithm ultimately,” Nitya says. “A lot of it has to do with how cleverly it is applied and how much thought is there behind the design of it.” In other words, while machines can do their best to figure out what someone wants, it’s up to humans to make a decision and contextualize their options.

Of course, shopping for dog food is a relatively trivial example of recommender systems in our daily lives, but it’s important to note that there can be a “dark side” of recommender systems. Recommender systems are both sociological and technical puzzles, according to Nitya. In some cases, recommender systems that constantly “reward” users for certain problematic behaviors can exacerbate polarization, addictive behavior (like incessant scrolling), alienation, and powerlessness, she explains.

Throughout the process of building a recommender system, Nitya suggests keeping these questions in mind: What user behavior is being rewarded here, and is it enhancing or diminishing the user’s quality of life? Are there consequences of scaling this recommender system that might cause harm down the line? In our courses, you’ll learn how to take these questions into consideration when building recommender systems, and be able to identify a “successful” one.

Ready to learn more about this fascinating machine learning application? Check out the beginner-friendly path Build a Recommender System to learn how to use the programming language Python to create a recommender system from scratch. If you already have experience with Python and Pandas, you might want to start with the free intermediate course Learn Recommender Systems.  
Be sure to check out Codecademy’s full catalog of machine learning and data science courses to learn more about the technology that you use every day.

Machine Learning Courses & Tutorials | Codecademy

Machine Learning is an increasingly hot field of data science dedicated to enabling computers to learn from data. From spam filtering in social networks to computer vision for self-driving cars, the potential applications of Machine Learning are vast.