Building AI on customer data: more data, a stronger model or a better signal?

Sensia Gen 8 · For AI and transformation leaders

When an AI system built on customer text disappoints, a programme usually reaches for one of three levers: give the model more data to read, buy a more powerful model, or improve how the data is understood and prepared. Each costs something different in money, time and skills. With the launch of Sensia Gen 8, the new generation of our qualitative intelligence engine, we examined the first two on our own production system, on a common Sensia preparation layer. Here is what we observed, and what it suggests about where to invest.

Three levers, three different bets

Reading more is the reflex answer to doubt: if the analysis missed something, give it more of the source. Buying power is the second: if the output lacks depth, use a stronger model. Both are easy to approve, because they look like a purchase rather than a project. Improving the data is slower and less visible: it means knowing, for each piece of text, what it is about and whether it answers the question.

The three are not interchangeable. They improve different things, at different costs, and they create different dependencies. A programme that cannot tell which lever moved which result ends up paying for all three. The benchmark examines how the system behaves when reading depth changes, and when the production configuration is compared with a larger-context reasoning configuration. Sensia's understanding and preparation layer provides the common foundation throughout.

How Gen 8 is organised

Sensia turns large volumes of heterogeneous verbatims into qualitative intelligence a team can use. The applications we present here concern the voice of consumers: product reviews, perceptions, needs, barriers, drivers of choice. The underlying problem is broader and familiar to any data team: understanding fragmented text, deciding what answers a question, measuring on qualified data and building a usable base of evidence at scale.

Gen 8 is a new generation of the whole engine, not only of one component. It combines the eighth generation of around twenty proprietary vertical models with advances in measurement, in the preparation and selection of material for each question, and in the use of much larger language-model context windows. Five functions follow one another; only one of them is a general-purpose language model.

The functions of the Sensia Gen 8 engine. Evaluation is a quality requirement applied to the output, not a separate product. How qualification and selection work remains proprietary.

Understanding is where we concentrate our own investment, because not every review speaks to a question. In our three reference portfolios, between 29% and 47% of the reviews collected carried a usable signal for the analyses; the rest had to be recognised and set aside before anything was written. Writing is where general-purpose models excel: synthesis, reasoning, a clear argument in any language. Reading a whole portfolio is not: it runs to many times the largest context window. Research by Chroma documented uneven use of long contexts and declining reliability as inputs grew in the models and tasks it tested (Chroma). The model vendors say much the same: Anthropic advises finding the smallest set of high-signal tokens for a task (Anthropic), and OpenAI presents fast, economical models as the right choice for well-defined tasks (OpenAI).

The experiments

We took five real production analyses on three of the largest and most varied portfolios we work on, in skincare and pet food, chosen for their size, fragmentation and diversity. Sensia's understanding and preparation layer provided the common foundation throughout. We then ran two experiments, for 122 generations in total. The first varies the reading volume with the same writing model: twenty sections at six amounts of material. The second compares two configurations on two sections, in which the model, the reasoning effort and the volume all differ, on a common Sensia preparation. Neither isolates the causal effect of each component. Each condition was generated once, so this is a production benchmark rather than a statistical study of repeatability.

To judge the results, we followed four things, each defined once here:

• Main findings: whether the conclusions a team would act on stay consistent, assessed by reading the versions side by side.
• Reach of the selection: the share of all relevant reviews on a question that are associated with the products present in the sample. It describes what goes into the analysis, not a percentage of insights.
• Depth of evidence: how detailed and well attributed the supporting evidence is: quotes, product references, level of competitive detail.
• Resources: the text passed to the language model, the LLM spend on the vendor's public rate card, and time.

Experiment 1: what does more material change?

The first lever is volume. With the same writing model, we regenerated twenty sections at six amounts of material: the sample our analyses used until now (about 160 to 280 reviews per section), about 500, 1,000 and 1,500 reviews, then a standard-window configuration filled to about 250,000 tokens (1,800 to 2,400 reviews) and a maximum-window configuration filled to about 900,000 tokens (up to about 9,000 reviews per section). Even the largest window is not a reading of a whole portfolio. The window names describe configured capacity; what was actually read is given in brackets.

Findings. Across the five analyses and six depth configurations, the main strategic findings remained consistent, from the initial samples to the maximum context window. This included the core opportunity territories, the leading choice driver and the central trade-offs in the analyses where they were assessed. Greater depth changed the supporting detail and presentation in some responses, without systematically revising the main strategic reading.

Reach. On four questions, we rebuilt the full pool of relevant reviews and measured the share associated with the products in each sample.

At around 1,000 reviews, selection reach ranged from 94.4% to 98.8% across the four questions; at around 1,500, it ranged from 98.1% to 99.5%. The progression differed by question. These results inform the choice of depth rather than establishing a universal sample size. Specific products or topics can require a targeted examination.

Depth of evidence. The benchmark also tracked product-level competitive detail, attribution and consumer quotations. Additional material provided further supporting evidence in some analyses: the usage-experience analysis, for example, contained 24 long quotations at the maximum window against 10 at around 1,000 reviews. The detailed observations distinguish this enrichment from a change in the main findings, and are reported in the benchmark report.

Resources. For the same twenty sections, the standard window doubled the LLM spend of the 1,000-review configuration, and the maximum window multiplied it by about thirteen, partly because every call crossed the vendor's long-context price threshold.

Our production configuration sits at about 1,000 reviews per section. That is a trade-off between a useful result, the depth each question needs, resources and a response time compatible with conversation, not a universal law: some questions justify more depth, and the extended windows remain available when they do.

Experiment 2: what does a high-reasoning configuration change?

The second experiment compared the production configuration with a high-reasoning configuration on two sections: opportunity territories and choice drivers. Both used evidence prepared by Sensia. The reasoning reference received a larger sample, around 230,000 input tokens or approximately twice the production input, and used high reasoning effort. This compares two system configurations; it does not isolate model capability alone.

Both configurations produced comparable main findings in the sections examined. The reasoning reference added some nuance in wording; no new main decision was observed. The resource comparison shows what each configuration consumed to produce that result.

The other resources point the same way, less dramatically. Across the two sections, the production configuration read 51.8% of the reference's input tokens and took between 31% and 45% of its generation time.

Gen 8 separates quantitative measurement from narrative synthesis. Counts and weights are computed on qualified review data; the language model develops the analysis from the prepared evidence. In the choice-driver comparison, the leading factor remained first in all seven versions, while the percentages written within the narrative varied. Those written estimates are distinct from the measured indicators used for quantitative reporting.

A separate observation: changing the writing model

A third comparison asks what happens when a new generation of model arrives. Newer does not automatically mean cheaper: GPT-5.5 doubled GPT-5.4's list prices, from $2.50 to $5 per million input tokens and from $15 to $30 per million output tokens (OpenAI, The Decoder). Yet Artificial Analysis reports that GPT-6.1 Sol costs around 64% less per benchmark task than GPT-5.6 Sol, while achieving a higher Intelligence Index score.

Source: Artificial Analysis, consulted 24 October 2026 (GPT-5.6 Sol, leaderboard). Cost per task is a weighted average over its own evaluations, including tokens, reasoning and caching; it is not a cost per successful task, nor a Sensia cost.

These are results on Artificial Analysis evaluations, not Sensia's workloads. The practical advantage is the freedom to adopt a model when it improves results, operating cost or response time on the application's own tasks, and to verify that improvement before relying on it.

GPT-5.4 was the model used in production before Luna. Because understanding, measurement and preparation live in our own engine, changing the writing model did not require rebuilding them. At standard rates, before caching, Luna's input and output tokens are both 12.5 times cheaper. Applying GPT-5.4's rates to the initial benchmark workload gives an estimated LLM cost of about $2.08 across twenty sections, compared with $0.166 for Luna. Even with the larger, roughly 1,000-review samples, Luna's cost was $0.527, about a quarter of that initial-workload reference. These are tariff comparisons, not measured historical production savings, and the benchmark did not regenerate the reports with GPT-5.4: the stability of findings it documents comes from the volume and reasoning experiments above.

Speed and conversation

Speed still matters because the product is used through conversation: users need to ask, refine and explore without losing the thread. That is different from answering instantly. It helps to separate four things: the initial preparation of the evidence, any preparation repeated for a new question, the generation itself, and the latency a user actually experiences in a conversation.

The benchmark measured preparation and section generation separately. Preparation time depended substantially on the implementation; the maximum-window tests used an adapted benchmark tool, so their preparation times cannot be read as a continuation of the production series. These measurements inform engineering choices, but they do not establish end-to-end conversational latency. The product requirement remains clear: users need to ask, refine and explore without losing the thread.

What this means for where to invest

Taken together, the experiments show consistent main findings across substantial changes in reading depth and in the writing configuration. Additional material expanded selection reach and enriched some supporting evidence; the reasoning reference added nuance at substantially higher LLM spend. These observations support Sensia's investment in specialised understanding, measurement and preparation as the foundation of the application. That foundation was held constant rather than independently tested in this benchmark.

The levers also combine. A cheaper writing model leaves room for deeper evidence: Luna at about 1,000 reviews per section costs roughly 3.9 times less than the GPT-5.4 tariff reference applied to the smaller initial workload. Those are two different configurations, not the same sample, and the comparison remains a tariff simulation at standard rates.

This is where the make-or-buy question becomes concrete. Synthesis is the capability most organisations will buy, from vendors investing billions to improve it; the design choice is to give it a defined role, so that each new generation can be adopted and verified rather than built around. Understanding the signal of a field takes years to build and constant work to keep current; it is worth deciding early whether to develop it in-house or source it from a specialist. MIT's 2025 study of enterprise generative AI reported that tools bought from specialised vendors or built through partnerships succeeded about two-thirds of the time, against about one-third for internal builds, and pointed to integration with real workflows as the main gap (Computing). Evaluation, finally, belongs to the business: it is the one capability no vendor can own for you.

Questions for your programme

About the tests: five production analyses (twenty sections) on three reference portfolios in skincare and pet food, regenerated at six amounts of material, plus a larger-context reasoning reference on two sections: 122 generations, October 2026. Reach measured on four questions through 24 read-only reconstructions, not counted in the 122 generations. The benchmark evaluates the Gen 8 engine in several configurations; it is not a controlled comparison with the previous generation. LLM spend on the vendor's public rate card; not the full cost of a programme. Full measures and results in the Gen 8 benchmark report.