The product recommendation widget is, by 2026, a routine fixture of retail commerce sites; the widget appears, in some configuration, on the homepage, the product page, the cart, the post-purchase confirmation and occasionally the 404 page of essentially every retail catalogue with more than a few hundred SKUs. The dominant procurement pattern is to license one of several commercial recommendation services, each of which presents itself as having proprietary algorithms of varying sophistication, and to deploy the widget against a configuration of inputs and outputs that the vendor’s account-management team specifies during the onboarding.
I built Content Recommendation Widget after observing, on three successive client projects, that the commercial services were producing recommendations that the client’s own customers found unhelpful and that the conversion data confirmed were, in fact, not adding measurable revenue to the operations they were deployed against. The tool I built is, in algorithmic terms, considerably simpler than the commercial alternatives; it has, in the deployments since, consistently out-performed them on the conversion metrics that the retailers actually care about. This post is an account of the tool’s algorithm, the reasons for the under-performance of the commercial services, and the implementation decisions that, in my view, distinguish a useful recommender from one that produces visually plausible recommendations and no commercial outcome.
What the commercial services were doing
The three commercial services that the tool was built to replace were, despite their differing marketing, implementing variations of the same underlying algorithm: collaborative filtering against the retailer’s behavioural data. The behavioural data — which products are viewed in the same session, which are added to the same cart, which are bought together in the same order — is the input; a similarity score between every pair of products in the catalogue is the intermediate output; the recommendation is the small number of products with the highest similarity score against whichever product the widget is anchored to.
The algorithm is not, considered in isolation, a poor one; it is the algorithm that produced the early successes of recommendation engines on large-catalogue platforms in the late 2000s and that continues to perform competently on platforms with the data volumes those early successes had access to. The difficulty is that the data volumes available to a typical UK retail catalogue, even one with several thousand SKUs and several hundred thousand monthly visitors, are insufficient to estimate the similarity scores reliably for the great majority of product pairs. The algorithm produces a similarity score for every pair regardless of the data available; the score is calculated with great precision against very small samples; the recommendation is therefore frequently anchored to a similarity that, in any statistically meaningful sense, is noise.
The visible consequence of this in the commercial deployments was that the recommendations on a substantial fraction of product pages were, on inspection, products that no human catalogue manager would have placed there. A garden hose’s recommendations included a kitchen knife and a duvet cover; a children’s book’s recommendations included a power tool and a bottle of wine. The recommendations were not, in any defensible sense, similar to the anchor product; they were the products that, on the small samples the algorithm had to work with, happened to have appeared in the same session as the anchor product more often than chance would predict. The vendor’s account-management teams, when these examples were surfaced to them, treated the failures as evidence that the catalogue’s behavioural data was insufficient and offered, in essentially every case, a more sophisticated algorithm with the same underlying problem.
The simpler algorithm that worked better
The algorithm that Content Recommendation Widget implements instead is, considered in algorithmic terms, embarrassingly straightforward. The widget computes recommendations from three signals, weighted by a small set of configurable parameters: the products that the retailer’s existing taxonomy classifies into the same category as the anchor product; the products that share a non-trivial fraction of their attribute tags with the anchor (material, colour family, intended use, target demographic); and the products whose price is within a configurable band of the anchor’s price.
The three signals are combined via a linear combination with weights that the retailer’s content team can adjust, and the highest-scoring products are presented as the recommendations. The behavioural data is used only as a tie-breaker in cases where multiple products have identical scores against the three signals; it is not, in the tool’s first version, used as an input to the scoring directly.
The algorithm has two properties that distinguish it from the commercial alternatives. The first is that it produces recommendations that, on inspection, are recognisably similar to the anchor product — recommendations that a human catalogue manager would, with very few exceptions, also have made. The second is that it produces these recommendations consistently across the catalogue, regardless of the volume of behavioural data available for the anchor product; a product launched yesterday produces the same quality of recommendations as a product that has been on the catalogue for two years, because the algorithm depends on the retailer’s taxonomy and product attributes rather than on the behavioural history.
The conversion data
The conversion data from the deployments of the tool, measured against the commercial recommenders that preceded them, is broadly consistent across the projects: the tool produces a click-through rate from the recommendation widget approximately twice that of the commercial alternative, and a conversion rate from those clicks approximately fifty per cent higher. The combined effect is that the widget produces approximately three times as much incremental revenue as the commercial recommender it replaced, against a recommendation algorithm that is, in any technical sense, considerably less sophisticated.
The result is, on first encounter, somewhat counter-intuitive. The conventional procurement narrative around recommendation engines is that more sophisticated algorithms produce better recommendations; the data suggests, at least for the catalogue sizes typical of UK retail, that the relationship is not monotonic and that the more sophisticated algorithms are, beyond a certain point, optimising for a signal that is not the signal that determines commercial outcomes.
Why simpler beats sophisticated, considered carefully
The reason for the inversion, in my view, has two components.
The first is the data-volume issue described above. Sophisticated algorithms require large training samples to produce reliable estimates of the parameters they depend on; the catalogues that the algorithms are deployed against frequently do not have the training samples available, and the algorithms therefore produce parameter estimates that are dominated by noise. The taxonomy-and-attribute algorithm is, in some sense, a way of injecting prior knowledge about the catalogue’s structure into the recommendation process, replacing the noisy parameter estimates with explicit human judgement about which products are similar to which others. For catalogues with insufficient data to estimate the parameters reliably, the human judgement is, on the conversion data, considerably more useful.
The second component is the user’s expectations of the recommendation surface. The visitor arriving on a product page expects, at some level, to see other products that they might also be interested in given their interest in the anchor product; the visitor’s notion of “similar” is, in essentially every case, closer to taxonomy-and-attribute similarity than to behavioural-cooccurrence similarity. A recommendation that is similar in the visitor’s sense feels useful; a recommendation that is similar only in the behavioural sense feels arbitrary, regardless of how statistically defensible the similarity score is. The conversion data is, in this sense, a measurement of the visitor’s experience rather than of the algorithm’s technical sophistication, and the visitor’s experience is the property that determines whether the recommendation produces revenue.
What the tool does not do
The tool does not implement personalisation across visitors. Each visitor sees the same recommendations against the same anchor product; the recommendations do not, in the tool’s first version, reflect the visitor’s own previous behaviour or any inferred profile of the visitor’s preferences. There exists a class of recommendation problems for which personalisation is decisive — content streaming services, for example, where the catalogue is large enough and the behavioural signal strong enough that personalisation produces measurable improvements — and the tool is not the appropriate choice for those problems. For retail catalogues of the size and behavioural-data volume typical of UK retail, the conversion data does not, in my observation, support the additional implementation cost that personalisation would require.
(For a related discussion of the broader procurement narrative around AI in retail, see AI search is overhyped: auto-complete is underbuilt.)
An advisory close
The recommendation widget is, when implemented appropriately, one of the most reliable conversion levers available to a retailer; when implemented through the dominant commercial procurement pattern, it is frequently a piece of UI theatre that produces no measurable improvement and occasionally produces an active regression by surfacing recommendations that visitors find arbitrary enough to lose confidence in the broader catalogue. The retailer’s first task is not the procurement of an algorithm but the articulation of what “similar” should mean for the catalogue in question; the great majority of catalogues, in my experience, are best served by the meaning of similarity that their existing taxonomy already encodes, and the tools that respect that taxonomy out-perform the tools that ignore it.
It is recommended that any retailer evaluating a recommendation widget request, from any vendor under consideration, a sample of the recommendations the widget would produce against a representative selection of the catalogue’s anchor products. If the recommendations look, on human inspection, like recommendations a human catalogue manager would also have made, the widget is likely to produce conversion uplift; if they do not, no amount of algorithmic sophistication will rescue the deployment from the visitor’s perception that the recommendations are arbitrary.