4 Principles To Make Your Recommendation Engine A Success

Meet the new players in the field of sharing suggestions and ideas—recommendation engines, or recommenders. Consider these four factors when making a bold move of building one for your brand.

Humans are inherently social creatures, and we advance collaboratively by proposing new ideas and sharing suggestions. Nielsen research from 2012 identified that 92% of consumers believe in recommendations from friends and peers over all forms of advertising.

The latest developments in this field are engines powered by people and software. One notable example is Stitch Fix—the tech and AI personal styling platform that has made almost $1 billion in sales and has just filed for IPO.

These systems are powered by software that analyses datasets and predicts likely matches. As I’ve written previously in this column, it’s the recent reduction in the cost of prediction that’s enabling more and more businesses to build such engines.

And we know recommendations keep customers engaged longer. A report into the state of the U.K.’s top 25 e-commerce fashion brands has revealed that the strongest sites offering product recommendations saw 140% more page views per visit and 10% higher task completion rates. Which is why the most ambitious brands are investing heavily in building, and improving these systems.

Best In Class

“Recommenders”—as they are often called—have been around since the 1990s. The Xerox innovation unit PARC published a research paper in 1992 titled “Using collaborative filtering to weave an information tapestry.” Google was founded in 1998, using the PageRank algorithm to power what is still the world’s largest recommendation engine.

In 2009, Netflix famously paid $1 million to an external team that was able to improve recommendations by just 10%. At the beginning of the competition, a dataset of more than 100 million ratings was released so it could be used as training data—an essential step in building and testing any engine.

More recently, Netflix has changed its ratings from stars to thumbs up/down—allegedly because the star-rating system was confusing to customers. It may also be the case that ratings were already grouped as like/don’t like, because people who give feedback tend to have strong positive or negative feelings. This can result in bimodal distribution, with two distinct groups emerging at either end of a five-star range.

Under The Hood

Building a recommender requires having solid data covering user behaviours, product information, and the reviews that link these two. Software is needed to create and manage the algorithms that make predictions, and some level of machine learning is needed to help your system evolve and improve.

The two most common techniques are collaborative filtering (predictions based on previous user feedback) and content-based (created by comparing product attributes). While user-based algorithms outperform product-based ones in most scenarios, they require past user data to get going—resulting in the common “cold start” problem.

Using a hybrid approach, combining collaborative and content-based filtering, can be more effective in some cases. Netflix is a good example of this as it compares the watching and searching habits of similar users, as well as offering movies that share characteristics with films that a user has rated highly.

Whichever approach you go for, you’ll need these four components:

1. Top-Class UI

A great customer experience is critical. After all, what’s the point in building a recommendation engine if no one can use it? Users must perceive suggestions as personal and helpful, and they must be able to use the recommendation features to give their own feedback. As with any digital interface, the elements should be tested with real customers, using CX measurement techniques fit for the digital age.

2. Rules For Prediction

An algorithm needs to be created to analyse available data and make predictions. Many systems use Bayesian ranking, although it should be used only to adjust ratings within families of comparable products.

The recency of feedback should be a key factor, especially with products that are updated and re-released. The quality of reviews is also important, and consideration should be given to the reputation of the reviewer, the quality of the written review, and whether reviews were “liked” or not. And all these factors need to be given individual weightings that reflect how you want the system to operate.

3. Humans And Machines Working Together

The software that runs recommenders has to be trained initially, and improved regularly, by humans. The people who perform these tasks also need to have strong skills to manage and communicate open-ended results and unexpected outcomes.

Stitch Fix asks customers to provide input through style surveys, measurements, Pinterest boards, and personal notes. Machine learning algorithms digest this unstructured information, and an interface communicates the results to the company’s 3,000 fashion stylists. These human experts select five items from a variety of brands to send out. The success of this approach is measured by the number of items returned by customers.

4. Strong Data Sources

Data is the key to the whole system, and needs to be stored in a way that can accommodate machine learning requirements. Extrinsic user feedback should be collected in the form of ratings, written reviews, and likes/dislikes. Intrinsic feedback should also be gathered, such as items viewed, content consumed, and product purchased. The more information you have about user behaviour (especially ratings) and items, the better your results will be.

Collecting new levels of data can have unintended benefits. Stitch Fix uses its customer data to develop new “frankenstyles”—fashions born entirely from data that are randomly modified over the course of numerous simulations.

What’s Next?

As machine learning becomes mainstream, systems with the capability to make predictions will be deployed to enhance experiences and retain customers.

Recommendation engines are on the rise, and are already helping the boldest of brands to differentiate in increasingly crowded marketplaces.