Key takeaways
- The unified layer answered at roughly 1/6th the token cost and half the tool calls of a five-connector stack with the same model and data.
- The unified layer used 6.8x fewer input tokens. Every connector adds its tool descriptions to every turn, whereas a single tool doesn't.
- Across 25 human-graded questions, the unified layer was better on 13 and the stack on 6. Its biggest edge was on everyday questions.
- The stack won on long writing tasks like release notes and deployment guides, where its longer, multi-source answers read as more complete.
- The unified layer returned an answer more often, 92.7% versus 87.2%, and fewer answers came back with nothing retrieved behind them.
We benchmarked a unified knowledge layer against five MCP connectors. Here's what happened.
Note: This is our own benchmark. Mosaic AI is the unified knowledge layer tested, and a Mosaic reviewer did the grading. Full method and limitations below.
Ask a support leader where their answers live, and you get the same answer: everywhere and nowhere. The knowledge exists. It sits in the helpdesk, chat threads, the CRM, a docs drive, a project tool. The problem was never that we lack knowledge. The problem is that we can't reliably retrieve the right piece at the right moment.
That retrieval problem is exactly what changes when you point an AI assistant at company data, and there are two very different ways to do it. You can connect the assistant to each source system separately and let the model figure out which tool to call and how to reconcile what comes back. Or you can put a single unified knowledge layer in front of it all and let the model ask one question.
We wanted to know which approach actually works better, so we ran a controlled benchmark. We used the same model, same data, same system prompt—the architecture was the main thing that changed.The benchmark captures the combined effect of that choice: the prompt weight it carries and the orchestration it asks of the model. This post walks through what we tested, how we ran it, and what the numbers said, including the places where the do-it-yourself stack held up better than we expected.
What we set out to test: The two architectures we compared
One common way to give an AI assistant access to company knowledge is to wire it to each source through the Model Context Protocol (MCP), a standard for connecting models to external tools and data. When Anthropic introduced the protocol in 2024, it described MCP as a way of "replacing fragmented integrations with a single protocol," the same fragmentation problem this benchmark set out to measure. You add one connector for the helpdesk, one for chat, one for the CRM, one for the docs drive, and so on. The model then decides which tool to call, in what order, and how to stitch the results together. We'll call this a generic MCP stack.
A unified knowledge layer takes the opposite approach. It ingests those same sources ahead of time, processes and enriches them into one searchable layer, and exposes a single natural-language tool to the model. The model asks one question and gets an answer grounded in everything at once.
Because both setups read from the same source systems, they answer from the same ground truth. That is what makes the comparison fair: the data is identical, and the only thing that differs is how it reaches the model.
How we ran the benchmark
We ran 109 real questions through both setups, paired question by question, and had a human reviewer grade a 25-question subset on answer quality. The questions were mostly real:
- 5 classic support-persona questions
- 7 classic go-to-market questions
- 97 sampled at random from what the team asked our in-house assistant over the prior one to two months.
We held everything constant except the connectors (MCPs). We used the same model (Claude Sonnet 4.6), the same published system prompt, and a fresh, memoryless conversation for every single question so no run could contaminate the next. Per-question limits, retries, and warm-up calls were identical across both setups. Both configurations ran directly against the Anthropic Messages API with the MCP servers attached to each request, not through any Claude product surface.
We also set the guardrails to work against us, not for us. Cost was calculated as a deliberately conservative upper bound that ignores prompt caching and batch discounts, so real-world spend sits at or below the figures below. We included only read-only tools from every connector, and scoped the source to what the authorizing user could see.
How much cheaper and lighter was the unified layer?
The efficiency results were not close. Across the full 109-question run, the unified layer answered each question with roughly one-sixth of the token cost and about half the tool calls of the MCP stack.
The 6.8x input-token difference is the mechanical heart of the result. Every turn in the MCP-stack setup carries the tool schemas for all five connectors, and multi-tool orchestration means more turns, each re-sending that context. The unified layer exposes one tool, so its prompts stay light, and it reaches an answer in fewer, cheaper turns. Output length barely differed between the two, which tells us the cost gap comes from input weight and orchestration overhead, not from one setup simply writing more.
Reliability pointed in the same direction. Neither setup produced a hard error. But the unified layer returned a non-empty answer on 101 of 109 questions, versus 95 of 109 for the stack, a 5.5-point edge in answers that actually came back with content.
Which setup gave better answers?
Efficiency is only half the story. A cheaper answer that's wrong isn't a win. So a human reviewer graded 25 paired answers on correctness, relevancy, and completeness. Grading was comparative and edge-based: a setup only had to be slightly better on a given question to count as a win, so the tallies capture whether there was a gap, not how large it was.
The overall row reads like a scoreboard. The unified layer took the edge on 13 of 25 questions (52%), the stack on 6 (24%), and the two were comparable or both weak on the rest. Of the 19 questions with a decisive winner, the unified layer won 13. Correctness was where both setups were most often correct; the separation showed up in relevancy and completeness, which is what you'd expect from a retrieval layer that surfaces the right material more reliably.
Where the edge came from, by question type
The most important nuance in the study is that the advantage was not evenly spread.
On the sampled questions, the real things the team asks day-to-day, the unified layer dominated, winning 9 of 13 with no bad losses. On GTM questions, which skewed toward long-form authoring, the stack actually edged the unified layer more often. The overall verdict favors the unified layer because real usage looks far more like the sampled set than like the authoring set.
What the differences looked like in practice
A few patterns showed up again and again across the runs.
The unified layer retrieved more sharply.The MCP stack more often issued broad or oddly-scoped queries and sometimes misread loosely related material as relevant.
The stack occasionally answered from priors with no retrieval at all. Asked "what is Mosaic," it made zero tool calls and described Mosaic as a monday.com feature, ending up substantially wrong, while the unified layer retrieved and answered correctly. On a data-retention question, the stack mixed source-specific details into an all-sources claim and cited a system no longer in use. These retrieval-free, confidently wrong answers are a concrete failure mode of leaving orchestration to the model.
It also cut the other way on authoring. Asked to draft release notes from the last two weeks of product updates, the stack was more correct and complete; the unified layer was too terse and missed recent items. That is an honest loss on a synthesis task, and it points to something real about how these architectures differ.
Where a unified layer wins, and where a stack still holds up
The pattern in the losses is worth naming, because it's a clue about what to build next. The questions the stack won were mostly time-bounded or status-specific: release notes from the last two weeks, roadmap items due this month. Those questions reward precise filtering on exact fields like dates and status, more than they reward semantic closeness.
A unified layer's retrieval today is based on semantic similarity, so it shines when the answer is "the most relevant thing across everything" and lags when the answer is "everything that matches these exact filters." Add SQL-style structured retrieval alongside semantic search, and a unified layer could filter on precise fields too, which is exactly what these authoring and time-scoped questions need. That is a forward-looking direction, not something reflected in these results, but it's the natural path to closing the remaining gap.
"You need a layer that turns unstructured data into structured signals." — Alon Talmor, CEO and Founder, Mosaic AI
What this means for connecting AI to your enterprise data
Read the results together, and a clear picture emerges. When you stitch five connectors together, you make the model the integrator. It has to decide what to call, absorb every connector's schema on every turn, reconcile conflicting results. That is where the cost, the latency, and the silent failures creep in. It's the classic build-versus-buy tradeoff: A do-it-yourself stack pushes the integration work onto you and onto the model.
A unified layer removes that burden. It does the integration once, at index time, so the model asks one question against clean, enriched context instead of orchestrating five raw feeds in real time. On identical data and an identical model, that single architectural choice drove a 6x cost gap, an 85% lighter token consumption, half the tool calls, and a higher rate of grounded, non-empty answers.
None of this replaces judgment about your own question mix. If your workload is heavy on time-scoped synthesis, a verbose multi-source setup may still read as more complete today. But for the retrieval questions most teams actually ask, unifying it up front was cheaper, lighter, faster to a good answer, and more reliable.
How Mosaic AI works: This is the bet behind Mosaic AI. Mosaic is an AI-native platform for technical customer support that unifies fragmented knowledge and data across 100+ tools into a single layer, then powers agent assist, ticket deflection, and case intelligence on top of it. It sits on the existing support stack rather than replacing it, so the model works from one enriched source of truth instead of orchestrating raw connectors on the fly.
How we're reading the limitations
A benchmark is only as honest as the caveats it names, so here are ours.
- Single model, single prompt. We did not sweep across model families or prompt variants. A differently configured stack could perform differently.
- Edge-based, single-reviewer grading. Quality was judged comparatively by one reviewer, so the tallies show direction, not effect size. The reviewer was an AI analyst who gave guidelines for comparative evaluation based on commonly used metrics (correctness, relevancy, completeness) and judged correctness based on internal company knowledge. Several correctness calls were cross-checked against internal research agents, but this is not a multi-rater study.
- The graded subset skews hard. Only 25 of 109 questions were graded, over-weighting the harder ICP and GTM sets. The cost and efficiency numbers use the full 109 and are not subject to that skew.
- Cost is list price. The figures are list-price, uncached, and unbatched. Caching and batch discounts would lower absolute spend, and their effect on the size of the gap is untested.
- One unadorned configuration. Both setups ran as a simple API baseline with the default prompt and no caching, skills, routing, or progressive disclosure. A more optimized stack could perform differently.
The efficiency and cost advantages are large and measured on the full run. The quality result is directional, consistent with equal-or-better overall and clearly better on real-usage questions, with authoring-heavy tasks as the honest exception.
Frequently asked questions
What is a unified knowledge layer?
A unified knowledge layer ingests data from multiple source systems. It processes, enriches and optimizes into one searchable index for retrieval, and exposes a single natural-language interface an AI model can query. Instead of the model calling a helpdesk tool, then a chat tool, then a CRM tool and reconciling the results, it asks one question and gets an answer grounded in everything at once. It is distinct from a raw connector, which simply exposes one source's data as-is.
What is a generic MCP stack?
A generic MCP stack connects an AI assistant to several source systems separately using the Model Context Protocol (MCP), a standard for linking models to external tools and data. Each connector exposes its own tools, and the model orchestrates across them, deciding which to call and how to combine what comes back. It is flexible, but it makes the model the integrator and inflates every prompt with each connector's tool schema.
Is Mosaic AI a unified knowledge layer or an MCP stack?
Mosaic AI is the unified knowledge layer—a single MCP in this comparison. It unifies fragmented knowledge and data across 100+ support tools into a single enriched layer, then powers agent assist, ticket deflection, knowledge automation, and case intelligence on top of it. Rather than making the model orchestrate raw connectors in real time, Mosaic does the integration up front so the model answers from one clean source of truth. To see it on your own data, book a demo.
