Skip to main content
DocumentationLibraryReference
On this page

Developers Command-line workflows

Test a shared model

List a model's tests, run them on the warehouse, read what failed, and keep the new rows after a change that you meant to make.

Audience
Data teams
Prerequisites
  • The Kater CLI, signed in with kater login
  • A local copy, made with kater git clone
  • kater debug passes

A test is a fixed question with a fixed answer: these fields, over these dates, return exactly these rows. Run a model’s tests after you change it, or after new data loads, so that no number moves by surprise. This page tests the shared model semantic.ad_reporting.

The commands and their times are real. Long waits are shortened in the recording.

  1. List the tests.

    kater test list shows each test of the model, its kind, and how it last ran. It runs nothing.

    Terminal window
    kater test list --model semantic.ad_reporting
    semantic.ad_reporting: 12 tests, last run 2026-10-16
      5 passing, 7 failing, 0 drafts, 0 never run, 7 stale
      2 refused, 7 rows, 3 same_as
    failed    clicks_impressions_by_platform        rows (stale)  added by Ana Lopez, 2026-10-16
    passed    spend_needs_currency                  refused       added by Ana Lopez, 2026-10-16
    failed    july_2026_totals_by_platform          rows (stale)  added by Ana Lopez, 2026-10-16
    passed    matches_dbt_ad_report_mart            same_as       added by Ana Lopez, 2026-10-16
    ...

    A rows test expects exact rows. A same_as test expects the same answer as another query, such as a dbt mart. A refused test expects the compiler to refuse its SQL, such as a total of spend across currencies. “stale” says that the test’s SQL moved since its last pass.

  2. Check without the warehouse first.

    --compile-only compiles every test and runs nothing on the warehouse. It says which tests your change moved.

    Terminal window
    kater test --select semantic.ad_reporting --compile-only

  3. Run the tests.

    Terminal window
    kater test --select semantic.ad_reporting

    Each failed test ends the run with a block: its name, its file, and what differs.

    15:02:10  Failure in test july_2026_totals_by_platform (connections/snowflake_prod_connection/modeling/kater_objects/tests/ad_reporting.sql)
    15:02:10    Got 6 values that differ from what the test expects.
    15:02:10    sql: 6 rows expected or returned on one side only.
    15:02:10
    15:02:10  Done. PASS=5 WARN=0 ERROR=7 SKIP=0 TOTAL=12

    Add --show-sql to print the Kater SQL and the warehouse SQL that each test ran.

  4. Keep the new rows after a change that you meant to make.

    When the numbers moved because you changed the model on purpose, --update records the new rows of each failing rows test and prints each number that moved. It never changes a same_as or a refused test. Read the numbers before you commit them.

    Terminal window
    kater test --select semantic.ad_reporting --update
    git diff

  5. Add a test.

    Write the question as Kater SQL in a file, and let --record run it one time and keep the rows that it returns.

    Terminal window
    kater new test july_2026_spend_by_platform --model semantic.ad_reporting --sql-file totals.sql --record

    The test goes in your own test file. Add --shared to put it in the shared layer’s test file, which everyone reads.

  6. Check that the model is ready to hand off.

    Terminal window
    kater test readiness --model semantic.ad_reporting
  • kater test list --model MODEL shows no test as “never run”.
  • kater test --select MODEL ends with ERROR=0.
  • Each number that --update moved is one that your change was meant to move.

If something looks wrong

You seeWhat to do
Got N values that differ from what the test expects. and the test’s SQL did not changeThe data changed. Check the source before you run --update.
The picker path does not compileThe test’s field picker selections are not valid on the model now, for example after a field was renamed. The message names the field. Change the test.
The run stops and names more than one connectionA run reads the tests of one connection. Add connection:NAME to the selection, such as --select connection:snowflake_prod_connection,tag:team=marketing.
A test is WARN with no answer yetIt is a same_as test with no native SQL, or a draft. Give it its other query, or run it one time with --record.

Next steps