You ask an AI tool for revenue last quarter. It answers in seconds, with a clean chart. You ask again tomorrow and get a different number. Which one goes in the board deck?
This is the main reason people stop trusting “chat with your data”. OWOX describes the problem: “The same question can produce five different SQL queries, each syntactically valid, each returning different numbers.” And the danger is that a wrong answer looks right: “the fabricated numbers look exactly like real ones.”
Ably’s product operations team saw the same thing at scale: “When anyone can get a confident, plausible-looking answer out of the data in seconds, every inconsistency in the data layer becomes a vector for a wrong decision made with high confidence.”
Why the AI guesses
Most AI analytics tools work the same way. You ask a question. The model reads your table names and writes SQL. The SQL runs. You see the result.
The model can write perfect SQL and still be wrong:
- It does not know your definitions. Does revenue include refunds? Is an active customer someone who logged in or someone who paid? The tables do not say.
- It decides again every time. Date boundaries, joins, filters and which column to sum: each is a choice. Ask twice and it can choose differently.
- It does the math itself. Some tools let the model add, divide or round numbers in its reply. A language model is not a calculator.
- Nothing checks it. The query runs, so it looks like a success. There is no definition to compare it to.
A better model does not fix this. The problem is that the model is asked to invent the business logic, not just the query.
The fix: give the agent definitions, not tables
The agent reuses governed metrics
In Kater, the data team defines each metric once, as a measure in Kater SQL. When your agent needs revenue, it finds that definition and uses it. It does not write a new formula. So the same question gets the same number, and that number matches every dashboard and app that reads the same measure.
Every figure comes from the warehouse
When an agent works in Kater, every figure comes from a query that ran on your warehouse, never from the agent’s own arithmetic. The query is the receipt. Results in chat arrive as cards: a chart for a result, and a row-by-row diff when you compare two.
In Slack, the Kater Slackbot answers with the figure, a link to the result in Kater and the date the data runs to. So you can always check where a number came from.
Kater checks every change
When an agent changes something, Kater compiles the write first and refuses it if it has an error. Every change has a diff, a History entry and an undo. An agent tries new work in a private brainstorm folder first, and asks before anything leaves it. Read more in BI as code.
The agent sees only what you can see
Your agent acts as the person who signed in, and sees only what that person can see. A question in Slack uses your own Kater access, so it cannot return data you are not allowed to read.
The agent and its rules are yours
Kater ships no agent of its own. You bring Claude Code, Codex, Cursor, Claude or ChatGPT, and connect it through the Kater MCP. Your rules, field descriptions and notes are files you own, so they are not buried under a vendor’s hidden prompt. We wrote about why that matters in Your instructions.md is losing to a prompt you can’t see.
What a trustworthy answer looks like
A good AI answer about your data has four parts:
- The number, from a query that ran on your warehouse.
- The definition it used, the same one every dashboard uses.
- The date the data runs to.
- A link to check it.
If an answer cannot show those, treat it as a guess.
Frequently asked questions
Why do AI analytics tools give wrong answers?
Most of them write new SQL for every question. The SQL can run without an error and still be wrong, because the model does not know what your company means by revenue or which rows to leave out. Ask again and it can make different choices and return a different number.
How does Kater make AI answers trustworthy?
The agent reuses metric definitions the data team owns in Kater SQL, instead of guessing SQL. Every figure comes from a query that ran on your warehouse, never from the agent’s own arithmetic. Kater compiles every write and refuses one with an error, and every change has a diff and an undo.
Does Kater have its own AI agent?
No. You bring Claude Code, Codex, Cursor, Claude or ChatGPT and connect it to Kater through the Kater MCP. The agent and its context are yours.
Can the AI see data I am not allowed to see?
No. Your agent acts as the person who signed in, and sees only what that person can see.
Want to ask your own data a question and check the answer? Book a demo.