Anthropic published a technical breakdown on June 3, 2026 detailing how its data team built a self-service analytics system using Claude that reaches 95% accuracy on business queries. The system handles routine ad hoc data requests from across the company, freeing the data science team to focus on forecasting, causal modeling, and strategic work.
What Happened
Anthropic deployed Claude as an internal analytics agent that queries production data systems and returns accurate answers to business questions. The result: 95% of incoming analytics queries handled without human data science involvement. The post documents the failure modes they encountered, the architecture that solved them, and the specific order in which their retrieval system operates.
What makes this notable is who wrote it. This is not a vendor case study or a research paper. It is a production post-mortem from the team that builds Claude, describing what broke when they tried to wire their own model into their own data stack and how they fixed it.
Why It Matters for Creators and Builders
Most teams attempting to use Claude or another LLM as a data analyst hit a ceiling quickly. The model picks the wrong column, hallucinates a metric name, or returns stale numbers because the business definition changed six months ago. These failures look random, but they are systematic. Anthropic identified them, named them, and published the architecture they used to eliminate them.
For creators running a content business, newsletter, or tool-based product who want to ask questions about their own performance data without writing SQL every time, this architecture is a directly applicable template, not a theoretical framework.
The Three Failure Modes (and How to Avoid Them)
The Anthropic team identified three root causes of LLM analytics failures. Understanding these shapes every decision in the solution stack.
Concept-to-entity ambiguity. A data warehouse contains hundreds of fields with overlapping names. When a user asks "how many active users did we have last week?", the agent must select the exact column matching the business definition of "active." If multiple candidates exist, the model guesses, and guessing produces wrong answers. The problem scales with warehouse size.
Data staleness. Business definitions change constantly. A metric called monthly_active_users may have been redefined multiple times. If the agent's knowledge of that metric comes from a description written at table creation, that description is wrong today. As Anthropic notes: "Data sources, business definitions, and schemas change constantly; assets and agent knowledge go stale and start returning subtly wrong answers."
Retrieval failure. Even with well-documented schemas, the search space in a production data warehouse is too large for unguided retrieval. The agent needs structured instructions about where to look first rather than semantic search across thousands of tables. This is where most RAG-on-data-warehouse setups fail by design, not by implementation. See the Anthropic tool use overview for how tool-based retrieval differs from raw context injection.
Anthropic's Three-Layer Analytics Stack
Anthropic's solution maps directly to the three failure modes. Each layer closes one of the gaps above.
Layer 1: Data foundations. The team enforced a single canonical dataset for each business concept. Where multiple tables could answer a question, one was designated authoritative and the others were deprecated. Metadata, meaning column descriptions, business definitions, lineage, and ownership, was treated as a production artifact maintained with the same discipline as application code. CI checks enforce accuracy. Stale descriptions fail review and block merges.
Layer 2: Sources of truth. The agent queries a three-tier information system in strict priority order:
- A semantic layer with pre-defined, human-curated metrics and dimensions
- Lineage and transformation graphs mapping current versus deprecated tables
- A corpus of historical SQL from existing dashboards and prior analyses
The semantic layer is the critical piece. Anthropic found that LLM-generated metric definitions "encoded the very ambiguities we were trying to eliminate." Human curation of metric definitions was not optional. The semantic layer must be maintained by people who understand the business, not generated automatically.
Layer 3: Skills-based routing. Rather than pointing Claude at raw warehouse data and hoping prompt engineering is enough, the team built reusable skill templates that encode business logic and direct the agent toward validated data paths. Skills act as structured system prompts that tell Claude which datasets are authoritative for which question categories, when to escalate to human review, and how to validate query results against known baseline values.
The full pattern is documented in the Anthropic Cookbook on GitHub, which includes templates applicable beyond analytics. For teams building on this foundation, the prompt engineering documentation covers the system prompt structures that slot into this skills layer.
How to Implement This in Your Own Workflow
You do not need an enterprise data warehouse to apply this architecture. The same principles work for a creator tracking newsletter performance, a product team measuring conversion, or a small agency reporting on client campaigns. Here is a practical starting sequence:
- Pick a single source of truth for each metric you track. If your subscriber count exists in three places, pick one as authoritative and document which one it is. Ambiguity at this level will surface every time you query Claude.
- Write and maintain metric definitions in plain language. Store them in a markdown file or a dedicated document Claude can read. When your definition of "engaged subscriber" changes, update the file immediately. This is your semantic layer.
- Build a small query corpus. Save 10 to 20 SQL queries, API calls, or spreadsheet formulas that correctly answer your most common questions. Add comments explaining what each one measures and under which conditions to use it.
- Write a system prompt that routes Claude to your canonical sources first. Name your metrics file and query corpus explicitly. List which data sources are current and which are deprecated. The specificity here is what separates 95% accuracy from 60%.
- Add online validation. Before returning an answer, instruct Claude to state which metric definition it used and which data source it queried. Spot-check these against known values for two to three weeks. This surfaces remaining ambiguities before they become habits.
The full Anthropic documentation hub includes additional patterns for grounding agents in structured data sources.
What This Enables
At Anthropic's scale, 95% accuracy on analytics queries means data scientists are no longer fielding ad hoc questions from product, marketing, and leadership teams. Routine queries go to the agent. Humans handle edge cases and the strategic work that requires judgment about causality and context.
For smaller operations, the return is faster. A solo creator or two-person team can get reliable answers to weekly performance questions without writing SQL or opening spreadsheets. The bottleneck is not building the agent. It is doing the data governance work first. That work is the same regardless of team size, and it is the part most teams skip.
Frequently Asked Questions
Does this require a data warehouse?
No. Anthropic uses a data warehouse, but the same architecture applies to a Postgres database, a set of CSVs, or API outputs. The key is maintaining clean metric definitions and a canonical source for each value, not the underlying storage system.
Why not just use an off-the-shelf LLM analytics tool?
Most LLM-powered analytics tools rely on column descriptions generated at table creation time. Without actively maintained metadata and human-curated semantic definitions, they hit the same staleness problem Anthropic identified. The differentiation in Anthropic's approach is treating metadata as a first-class production artifact with CI enforcement, not a documentation afterthought.
What makes Claude specifically well-suited for this use case?
Claude's long context window allows a complete semantic layer document, lineage graph, and historical query corpus to be held in a single context. For small to medium deployments, this eliminates the need for a separate vector database to handle retrieval. The three-tier priority system operates within the same context window that holds the system prompt and conversation history.
How are queries handled when no canonical metric exists?
Anthropic routes these to human review. The agent flags queries it cannot answer with high confidence rather than producing a plausible-looking but unreliable result. Explicit uncertainty reporting is built into the system as a feature, not treated as a failure. This uncertainty flag is part of how 95% accuracy is maintained rather than degraded over time.
Where can I find Anthropic's skill template?
Anthropic includes a reusable skill template in the appendix of their original blog post. It provides a structured format for encoding business logic and routing instructions across different query categories.
