Beyond classic RAG: hybrid GraphRAG for ecommerce recommendations that convert

In July 2025, a University of Leeds team ran 600 questions through three retrieval pipelines, and vector-only RAG — the architecture inside most ecommerce recommendation engines — came out with a factual correctness score of 0.48. Coin-flip territory.

The same benchmark gave hybrid GraphRAG 0.58

Vector-only RAG finds text that looks similar to a shopper's query, but it cannot reason about how products, policies and past orders relate. Hybrid GraphRAG — vector search fused with knowledge graph traversal — lifted factual correctness by 8 percentage points in the Leeds benchmark, and an agentic variant built at Duke University added +0.63 context recall by letting an agent choose the retrieval mode per query.

This stack: hybrid retrieval, multi-agent orchestration, citation layers, and a propensity-to-buy re-ranker.

Where vector-only RAG breaks down

Vector search fails on exactly the queries that carry revenue. Alhena AI, which runs a hybrid stack in production for brands like Tatcha and Puffy, lists three: multi-hop reasoning ("can I return the gift my sister bought 45 days ago under your holiday policy?"), entity disambiguation (a SKU versus a variant versus a bundle), and relationship queries ("which accessories are compatible with model X?"). None of these is a text-similarity problem.

A chunk of policy prose doesn't encode the connection between an order, a person, a time window and a rule — so no embedding model will ever rank it correctly for that question.

The Leeds benchmark puts numbers on the failure shape: vector RAG scored 0.61 on easy, single-fact questions — the best of all three pipelines tested — then fell apart as questions demanded multi-hop reasoning, ending at 0.48 overall factual correctness against 0.58 for hybrid GraphRAG. Faithfulness told the same story: 0.55 for vector-only, 0.59 for both graph variants.

Here's the part nobody quotes: on easy queries, the graph pipelines were worse.

If your traffic is mostly "show me a lightweight rain jacket for hiking," classic vector retrieval already does the job, and adding a knowledge graph adds cost without adding accuracy.

The case for GraphRAG rests on the hard 20% of queries — compatibility, policy, order-aware questions — which happen to be the queries closest to a purchase decision.

GraphRAG: your catalog is a graph, not a bag of chunks

GraphRAG (retrieval over a knowledge graph of typed entities and relationships, instead of text chunks) models the catalog the way your merchandising team already thinks about it. Products, variants, brands, suppliers, reviews, policies — nodes.

The edges do the work: COMPATIBLE_WITH, BELONGS_TO, GOVERNED_BY, SUPERSEDES.

Ask "is this serum compatible with the retinol I bought last month?" and the graph traverses one typed edge instead of hoping two chunks embed nearby.

Construction is LLM-driven extraction: the Leeds team used Neo4j's LLM Knowledge Graph Builder with LLMGraphTransformer to turn raw documents into nodes and edges, stored in Neo4j AuraDB and queried with Cypher.

The unglamorous truth is that extraction is the easy half. Entity resolution — deciding that "Retinol 0.5%", "retinol serum 0.5" and SKU 44821-B are the same node — is where GraphRAG projects go to die. Budget for it.

Now the controversy: most GraphRAG tutorials let the LLM write Cypher queries at runtime.

Don't. Alhena refuses to: they query a vector index over entity-node embeddings, then expand matched entities into their relationship subgraph — no generated database syntax, no hallucinated schema references.

The Duke results back the caution: their agent's Cypher generation was the one place metrics dropped after fine-tuning (KG precision −0.04, KG faithfulness −0.03). Debugging a bad generated Cypher query at 2 a.m. is a special kind of pain. Constrain the graph surface with prevalidated query templates and let the model pick parameters, not syntax.

Multi-agent orchestration: route the query, don't marry the pipeline

The strongest 2025 result in this space isn't a better retriever — it's an agent that picks the retriever. Nagori's team at Duke wrapped both pipelines in a Llama-3.3-70B agent that decides, per query, whether to run GraphRAG over the Neo4j knowledge graph or VectorRAG over FAISS, then quantifies its own uncertainty at runtime. After Direct Preference Optimization on 15 human-annotated preference pairs — fifteen, not fifteen thousand — the agentic system beat the non-agentic baseline by +0.63 on vector-store context recall and +0.56 on overall context precision, with +0.24 on vector-store faithfulness.

For ecommerce, the same pattern maps to a small cast of specialists behind one orchestrator: a discovery agent for open-ended browsing, a compatibility agent that lives on the graph, a review-synthesis agent, a policy agent wired to order management APIs. Tools exposed via MCP, queries routed by intent.

Take the contrarian warning seriously, though: start with a router, not a swarm. A single orchestrating agent with two or three tools captures most of the gain the Duke paper measured. Four cooperating agents with cross-validation — like the LangGraph + Neo4j "Research Council" system that circulated on Reddit in March 2026 — add confidence scoring and cited verdicts, and also add latency, failure modes and token spend. Earn the complexity.

Trust is the conversion feature

Every claim the assistant makes should trace to a chunk or a graph node, visibly.

Alhena runs three verification layers on top of retrieval: source attribution on every response, deferral to a human when no grounded source exists, and conflict detection when new FAQs contradict existing knowledge. Their framing is honest in a way most vendors aren't — "hallucination-resistant, not hallucination-proof."

This is where the metrics stop being academic. A recommendation that cites the actual review, the actual compatibility edge and the actual return policy converts differently than a confident paragraph with no provenance. Citations build trust. Trust builds conversion. And the return rate drops, because "the assistant said it would fit" stops appearing in support tickets. The faithfulness gap in the Leeds data — 0.59 versus 0.55 — looks small until you price it across a million sessions.

Propensity-to-buy: the re-ranker that pays for the GPUs

Retrieval answers "what's relevant to this query." It says nothing about "what is this shopper likely to buy." That second question is a scoring problem, and it belongs in the ranking layer, not the prompt.

The pattern we've run in our deploys: compute a propensity score from session behavior, purchase history, stock levels and margin, then blend — final rank = relevance score weighted against propensity, with explicit guardrails so high-margin items can't crowd out genuinely relevant ones. The guardrail matters more than the formula.

Without it, the re-ranker quietly becomes a margin maximizer, shoppers notice the recommendations getting pushier, and the trust you bought with citations evaporates. The best weighting we've used held relevance dominant at roughly 70/30, though your mileage will vary with catalog size and traffic mix.

This is also where the graph pays a second dividend: propensity features like "bought the compatible base product" or "returned a sibling variant" are edge traversals, not feature-engineering archaeology.

Classic vs hybrid vs GraphRAG multi-agent

Case study: Alhena AI in production

Alhena AI runs the architecture this article describes — vector store and knowledge graph fused in one pipeline, agentic layer on top, live commerce tools (catalog APIs, order management, policy engines) alongside. Their reported outcomes: Tatcha at 3x conversion rates from AI-assisted shopping, Puffy at 90% CSAT with 63% of conversations resolved without a human. Deployment claim: under 48 hours, with a configuration tuner that infers domain, entity types and relationship patterns from the catalog instead of hand-built ontologies.

Two honest caveats. These are vendor-reported numbers with no public A/B methodology, so treat them as directional, not gospel. And the Leeds benchmark documents the real cost of the hybrid approach: context relevance dropped to 0.04 for hybrid GraphRAG versus 0.11 for pure graph retrieval, because fusing two retrieval legs drags in verbose, loosely relevant context. The fusion layer needs its own pruning pass, or you trade hallucination for noise.

The build order

1. Ship hybrid search first — BM25 plus vectors plus a reranker, with proper score normalization before fusion.
2. Stand up an eval harness before touching the graph: RAGAS metrics (faithfulness, answer relevance, context precision and recall) on 50–100 real shopper queries.
3. Build the knowledge graph for the top 20% of categories by revenue only — the ones where compatibility and policy questions actually move money.
4. Add a router agent that picks vector, graph or both per query. Resist the swarm until the router is stable in production.
5. Turn on citations: every recommendation traces to a chunk or node, and the agent defers when nothing grounds it.
6. Layer the propensity re-ranker on top, relevance-dominant, with margin guardrails.
7. A/B against the old pipeline on CTR, conversion, average order value and return rate — not on demo queries.

What it costs to run

A routed multi-agent recommendation burns 4–8 LLM calls per shopper query — routing, retrieval decisions, synthesis, verification.

The Duke team measured roughly 2 minutes per query on consumer hardware and about 10 seconds on GPU-backed infrastructure, which tells you where the latency budget goes. Cost control is an architecture decision: semantic routing sends the easy 80% of calls to small models, response caching absorbs repeat catalog questions, and frontier models see only the final synthesis step.

One more constraint if your shoppers are in the EU: behavioral data feeding propensity scores sits squarely under GDPR, and recommendation systems are in scope of the EU AI Act's transparency provisions. Running inference on EU-sovereign infrastructure — which is the problem we work on at Regolo.ai — removes a compliance conversation you'd otherwise have with every enterprise prospect. It also, frankly, simplifies the data-processing agreement stack.

FAQ

Is GraphRAG worth it for a small ecommerce catalog?
Usually no. Under a few thousand SKUs with simple compatibility rules, hybrid search covers most queries at a fraction of the cost. Graph construction pays off when relationship queries — compatibility, policy, bundles — drive a meaningful share of revenue.

What's the difference between hybrid search and GraphRAG?
Hybrid search combines keyword and vector retrieval over text chunks, then fuses and reranks. GraphRAG retrieves from a knowledge graph of typed entities and relationships, which lets it answer structural questions that chunk similarity cannot resolve.

How do I stop my RAG recommendation bot from hallucinating product details?
Ground every claim in a retrieved chunk or graph node and show the citation. Add a verification pass before rendering, and let the agent defer to a human when no grounded source exists. Graph-based pipelines scored 0.59 on faithfulness versus 0.55 for vector-only RAG in the Leeds benchmark.

How much does a multi-agent recommendation system cost to run?
Plan for 4–8 LLM calls per recommendation with a routed design. Semantic routing to smaller models, response caching, and reserving frontier models for final synthesis keep the bill proportional to the revenue the recommendations generate.

Can I add propensity-to-buy scoring to an existing RAG pipeline?
Yes — it's a re-ranking layer, not a rebuild. Blend a propensity score (session behavior, history, stock, margin) with the retrieval relevance score, and add guardrails so margin doesn't crowd out relevance.

Customize your RAG and keep private your Customer data, ask me how