Building a Real-Time Product Recommendation Engine: Architecture Notes

"Just add AI recommendations" is one of those requirements that sounds like a single ticket and turns into three separate systems once you actually scope it: a data pipeline, a model layer, and a real-time serving layer, each with its own failure modes. Here's a breakdown of how the pieces actually fit together, based on patterns that show up repeatedly in production ecommerce recommendation systems.

The three algorithm families, and when each one breaks

Collaborative filtering recommends based on behavioral similarity — users who bought A also bought B. It doesn't need to understand why products are related, which is its strength and its weakness. It breaks down hard on the cold-start problem: a new product with zero purchase history has no signal for the model to work with, and a new user with no history gets generic, unpersonalized results until enough interactions accumulate.

Content-based filtering recommends based on product attributes — category, price range, tags, description embeddings. It handles cold-start for new products fine, since a new item still has metadata to match against. Its weakness is the opposite: it tends to over-recommend near-duplicates of what a user already looked at, and struggles to surface genuinely novel but relevant items outside a user's established pattern.

Hybrid models combine both, typically by blending scores from each approach or by using content-based filtering specifically to cover the cold-start gap while collaborative filtering handles established users and products. Most production systems end up here — a pure single-method system is usually a temporary state, not a destination.

The real-time serving problem is a different problem than the model

A model that returns good recommendations in a batch job overnight is a completely different engineering challenge than one that has to respond to "user just added item X to their cart" within the same page render. Three practical implications:

• Pre-compute what you can, and compute the rest on read. Full model inference on every request doesn't scale past a small catalog. Most production systems pre-compute candidate sets (e.g., "items similar to X" for every product) as a batch or near-real-time job, then apply lightweight re-ranking at request time based on the specific session's live signals.
• Session state needs to update without a full model re-run. A simple, effective pattern: maintain a short-lived session vector (recently viewed/added items) that gets blended with the precomputed candidate set at serving time, rather than re-running the full recommendation model per click.
• Latency budgets matter more than model sophistication past a certain point. A slightly worse model that responds in 50ms beats a better model that adds 400ms to page load — recommendation quality gains get eaten by cart abandonment from a slower page.

A minimal architecture sketch

[User Events] to Event Stream (Kafka/Kinesis) to Feature Store, which feeds both a Batch Job for candidate generation and a Real-time re-ranker. The Batch Job produces Precomputed candidates, which flow into the Serving API alongside the Real-time re-ranker's output. The Serving API feeds the Frontend widget.

The split worth internalizing: candidate generation (expensive, can be batch/async) is architecturally separate from candidate ranking (cheap, must be real-time). Conflating these into one synchronous call is the most common reason a recommendation feature becomes the slowest part of a page.

Privacy-by-design isn't optional overhead — it's a schema decision

If you're building this for a market with strict data protection requirements (Australia's Privacy Act 1988, GDPR, or similar), the cleanest approach is separating raw behavioral logs from the derived feature vectors used for inference. Raw event logs (what a specific user clicked, in order, with timestamps) carry far more re-identification risk than an aggregated feature vector, and they typically have different retention and consent requirements.

Practical pattern: raw events go into a short-retention store used only for feature computation; the resulting feature vectors (which are much harder to reverse into individual behavior) feed the actual model, and those are what get retained longer-term. This isn't just a compliance checkbox — it also tends to produce a cleaner separation between your ingestion pipeline and your model layer, which pays off in maintainability independent of the regulatory motivation.

The feedback loop is the part most implementations skip

A recommendation model that doesn't retrain on its own outcomes will drift — user behavior shifts, catalogs change, and a model trained once on historical data slowly loses relevance. The minimum viable feedback loop needs three signals tracked explicitly, not just inferred:

• Impressions — what was actually shown (you can't measure lift without knowing the baseline exposure)
• Interactions — clicks, adds-to-cart, on the specific recommended items, not just general site activity
• Outcomes — completed purchases attributable back to a recommendation, which usually requires attribution logic more careful than "last click before purchase"

A/B testing the ranking logic itself (not just whether recommendations exist at all) is where most of the ongoing value comes from post-launch — placement, count of items shown, and re-ranking weight between recency and relevance all move conversion independently of the underlying model's raw accuracy.

Where this tends to go wrong in practice

Treating it as a one-time model training exercise. The model degrading over time without a retraining and monitoring pipeline is the most common reason a promising launch quietly underperforms six months later.

No fallback for the cold-start and empty-candidate cases. What renders when a user has no history and a product has no purchase data yet? A recommendation widget that silently fails or shows nothing is worse for UX than a well-designed non-personalized fallback (trending items, category bestsellers).

Over-indexing on model sophistication before the serving architecture can support it. A hybrid deep learning model that can't return results within the page's latency budget delivers less real value than a simpler model that actually ships and runs fast.

I wrote a longer, business-focused version of some of the AU-market specifics (privacy law considerations, local case study data) here: https://7pillars.com.au/blog/ai-technology-in-ecommerce-personalised-product-recommendations/ — this post is the more technical/architectural companion to it.