Skip to main content
On this page

Model Trust

Tests

Pin known answers for a model, so a change to the model, its data or Kater never moves a number by surprise.

Audience
Data teams
Prerequisites
  • A semantic model or Kater dataset
  • A number you trust to check against

A test is a fixed question with a fixed answer: these fields from this model, over this date range, return exactly these rows. Write a small suite when a model is done, check its numbers against something you trust, and run it whenever the model, its upstream data or Kater changes. Each model’s tests are in the Tests tab of the Shared models editor.

The Tests tab

Open a model in Shared models and select the Tests tab in the sidebar. It shows:

  • Tests and how many the model has, with New test (+) and Run every test (the run button).
  • Readiness. Not ready to hand off or Ready to hand off, then each readiness check with its detail, such as “0 of 16” for measures in a checked test.
  • The model’s tests, each with its state, its name and its kind: Rows, Refused or Same as.

A model with no tests yet offers Generate drafts and New test. Under Business questions with no test, each of the model’s business questions has Make a test from it, which writes one draft test that answers that question.

The Tests tab for Marketing Sessions with 0 tests: Not ready to hand off with Every test passes, No tests yet, Every measure in a checked test, 0 of 16, and Business questions answered, 0 of 4, then Generate drafts, New test, and four business questions with no test, each with Make a test from it
The Tests tab of a model with no tests.

A test, opened

Select a test in the list, and it opens in the main area. The bar at the top has Joins and rows to go back, the test’s state and name, its author, Show in Files, Record rows and Run. A test with no description says “No description yet. Add what you checked these numbers against.” Below it is the test’s block, as the file has it, which you edit in place. Under the block:

  • Rows shows the expected rows, and after a run, what the run found.
  • Paths shows each path the test checks: the picker request, the SQL the writer writes, and the other side of a same_as test.
The clicks_impressions_by_platform test of Ad Reporting opened in the editor: Joins and rows, Not run, Show in Files, Record rows and Run, the note No description yet, its block from line 205, and Rows with the recorded expected rows for facebook_ads and tiktok_ads. The Tests tab at the left lists six readiness checks and 12 tests
A test opened in the editor.

Files keeps its own Tests view, with every test in the connection.

Where tests live

A model’s tests live in one file named after the model. tests/ad_reporting.sql holds the tests of semantic.ad_reporting.

WhereWho reads itWho writes it
modeling/kater_objects/tests/<model>.sql, the shared testsEveryone who reads the model. Only these count for readiness and the nightly run.People who edit the shared layer
users/<email>/tests/<model>.sql, your ownYou, anyone you share it with, and adminsYou. New tests go here unless you ask for the shared file.
workspaces/<w>/tests/<root>.sqlReaders of a connection workspace. These test the root’s filters and parameters.People who edit the workspace

Example

modeling/kater_objects/tests/ad_reporting.sql
---
name: july_2026_totals_by_platform
model_type: test
description: Matches the Ads Manager exports for July 2026.
expect:
columns: [platform, union__currency, sum__spend, sum__clicks]
rows:
- [facebook_ads, USD, 63963.91, 58844]
- [google_ads, USD, 110747.05, 79509]
- [tiktok_ads, USD, 12535.44, 19704]
---
SELECT
ad_reporting.platform,
ad_reporting.union__currency,
ad_reporting.sum__spend,
ad_reporting.sum__clicks
FROM semantic.ad_reporting AS ad_reporting
WHERE
KATER__IS_BETWEEN(ad_reporting.union__date_day, CAST('2026-07-01' AS DATE), CAST('2026-07-31' AS DATE))
GROUP BY
ad_reporting.platform,
ad_reporting.union__currency

Each test is one block with its own frontmatter. A test compiles as a query in a workspace rooted in the model, so grouping, KATER__ conditions and currency rules apply exactly as they do for a stakeholder’s query. Nothing reads a test: it is not in pickers, the catalog or Explore.

Keys

KeyWhat it doesExample
nameRequired. Unique within the file.spend_needs_currency
model_typeAlways test.test
descriptionWhat you checked the answer against.Matches the Ads Manager exports for July 2026.
tagsTags for selecting tests with --select tag:KEY=VALUE.
modelOnly when a Kater dataset and a semantic model share the file’s name.semantic.ad_reporting
questionThe model’s business question the test answers, word for word.
drafttrue until you have checked the answer.true
as_ofThe day the test takes as today. It pins relative dates.'2026-08-03'
inputsWorkspace tests only: the parameters and the optional filters to switch on.filters: [reporting_date]
givenA fixture: rows to use in place of each table the model reads.See below
pickerThe question as field picker selections: fields, conditions, group, order, limit.group: true
SQL bodyThe question as hand-written Kater SQL. A test has a picker, SQL, or both.SELECT ...
expect.columns, expect.rowsThe exact answer.[google_ads, USD, 110747.05, 79509]
expect.same_asAnother query whose answer must match, joined on key columns. native is read-only warehouse SQL, sql is Kater SQL.key: [platform]
expect.refusedThe diagnostic code the compile must fail with.not_additive_across
expect.toleranceAn absolute tolerance by column, where the format’s precision is wrong.sum__spend: 0.05

Two paths, three expectations

  • A picker tests what a stakeholder’s clicks produce. Kater writes its SQL at each run, so the test also checks the SQL writer.
  • A SQL body tests hand-written Kater SQL. With both, each path is checked on its own.

A refused test keeps a rule from being lost. This one proves that Spend cannot be totalled across currencies:

modeling/kater_objects/tests/ad_reporting.sql
---
name: spend_needs_currency
model_type: test
expect:
refused: not_additive_across
---
SELECT
ad_reporting.platform,
EVAL(ad_reporting.sum__spend)
FROM semantic.ad_reporting AS ad_reporting
GROUP BY
ad_reporting.platform

A same_as test checks the model against a source you trust, such as a dbt mart:

modeling/kater_objects/tests/ad_reporting.sql
expect:
same_as:
key:
- platform
- union__account_id
- union__date_day
native: |
SELECT
platform,
CAST(account_id AS VARCHAR) AS union__account_id,
DATE_TRUNC('MONTH', date_day) AS union__date_day,
SUM(spend) AS sum__spend
FROM DEMO_DB.ANALYTICS_AD_REPORTING.AD_REPORTING__AD_REPORT
WHERE date_day BETWEEN '2025-05-01' AND '2026-07-31'
GROUP BY 1, 2, 3

Every test pins a fixed set of rows

Each side of a test filters the model’s date to the same closed range, so the answer does not move as data loads. A test without one is refused, except a refused test or a fixture test.

  • A closed range. A picker condition is_between, a BETWEEN in SQL, or KATER__IS_BETWEEN.
  • A relative window, pinned. is_within_last with complete: true, and as_of to fix today.
  • Settled data. A range that ends within 30 days of today gets a warning, because ad platforms restate recent conversions.

A fixture test runs the model on rows you give, as a dbt unit test does. Every other table it reads is empty:

modeling/kater_objects/tests/ad_reporting.sql
name: account_with_two_currencies_counts_spend_twice
model_type: test
given:
- table: facebook_ads__account_report
rows:
- {SOURCE_RELATION: fb, DATE_DAY: '2026-07-01', ACCOUNT_ID: 1, CURRENCY: USD}
- {SOURCE_RELATION: fb, DATE_DAY: '2026-07-01', ACCOUNT_ID: 1, CURRENCY: EUR}
- table: facebook_ads__ad_report
rows:
- {SOURCE_RELATION: fb, DATE_DAY: '2026-07-01', ACCOUNT_ID: 1, AD_ID: 10, SPEND: 100, CLICKS: 7}
expect:
columns: [platform, union__currency, sum__spend, sum__clicks]
rows:
- [facebook_ads, EUR, 100.0, 7]
- [facebook_ads, USD, 100.0, 7]

Write tests

  1. Generate a starting suite.

    In the Tests tab, select Generate drafts, or Model › Generate tests. To test one business question, select Make a test from it under the question. From a terminal, kater test generate --model semantic.ad_reporting writes five draft tests, one of each shape, into your own file. Add --shared for the shared file, --count 12 for more coverage, and --from-questions to turn the model’s business questions into tests.

  2. Or write one yourself.

    Select New test in the Tests tab, or New › Test. In the workspace editor, select Query › Add as test, then enter a Test name.

  3. Check each draft.

    Open it and select Run. Compare its rows with a number you trust, and write a description that says what you compared with. Then mark it checked: kater test update NAME --model semantic.ad_reporting --checked.

  4. Add the reference test.

    Write the native side of the same_as test against your source of truth. Kater never writes native SQL, so that test reports incomplete until you do.

From a terminal, kater new test NAME --model semantic.ad_reporting --sql-file totals.sql --record runs the SQL once and keeps its rows. --refused keeps the code the compiler refuses it with, and --same-as-native mart.sql --key platform writes a reference test.

Run tests

  • In the editor. Run every test in the Tests tab, or Model › Run tests, runs the model’s tests. Run on an open test runs that one. Record rows, or Model › Record test rows, keeps what the run found as the expected rows.
  • From a terminal. Use the commands below.
CommandRuns
kater test --select semantic.ad_reportingEvery test of the model you can read.
kater test --select state:modified+ --state git:main --staleThe merge check: only tests a change could reach, and only those whose compiled SQL moved since their last pass.
kater test --select state:modified+ --state git:main --compile-onlyCompiles the tests a change could reach, without the warehouse. Good for a commit hook.
kater test --updateRecords new rows for failing tests after an intended change, and prints each number that moved.
kater test snapshot, then kater test --localCopies the rows tests read into a local DuckDB file, then runs on it with no credentials. A local run never replaces a warehouse run.

Columns compare by position, and numbers at the precision their format shows, such as cents for currency(…, 2). A failure whose SQL is unchanged since its last pass means the data changed, and the test says so.

Ready to hand off

The top of the Tests tab says whether a model is ready for stakeholders, and lists each check with its detail. kater test readiness --model semantic.ad_reporting gives the same answer. Readiness only informs. Nothing is ever blocked for a model without tests.

CheckMet when
Every test passesEvery shared test of the model passes.
No draftsNo test is still a draft.
Every measure in a checked testEach measure of the model is in a checked test, such as “5 of 5”.
Matches native SQLA same_as test against native SQL, such as a dbt mart, passes.
Every rule pinnedEach declared rule has a refused test. The detail names the rules still missing, such as “Spend without Currency”.
Business questions answeredEach of the model’s business questions has a checked test that answers it.

Run them every night

A test job in kater.config.yaml with after: its sync job runs the shared and workspace tests after each sync, so restated data shows as failing tests in the header’s test bell. See Scheduled jobs.