kater.config.yaml holds your organisation’s settings for scheduled jobs, saved query results and queries on the warehouse. It is one file at the root of your content, beside connections/. Every key is optional. A key you leave out takes its default, and so does every key when the file is missing.

What reads it

• Changes apply at once. Kater reads the file each time it needs it, so a change needs no restart.
• Kater checks it when it reads it. A save or a publish does not check it. See When the file is not valid.
• A new project starts with it. Onboarding writes the file with no jobs, as jobs: {}. When the content store already has the file, onboarding adds no starting files.

Who can change it

• Organisation admins. People with access:administer, which the Admin role has, can change it.
• Editors of the content root. A person with Editor or Manager on the root can change it too. New organisations give Everyone only Viewer there.
• Everyone can read it. The root starts with Everyone as Viewer, so keep secrets out of the file. A job gets its secrets from --env-file. See Scheduled jobs.
• Kater reads it as an admin. Every query run and every job gets the same settings, whoever starts it.

Change it

kater git clone ~/kater
cd ~/kater
# Edit kater.config.yaml, then publish it
kater git commit -am "Keep query results fresh for 1 hour"
kater git push

Example

This is ShopTech’s file, with cache and queries added. It syncs the Snowflake connection at 02:00 UTC once the schedule is on, then runs every test of the connection.

1
jobs:
2
  sync_snowflake:
3
    command: kater sync
4
    connection: snowflake_prod_connection
5
    deployment: production
6
    schedule:
7
      enabled: false
8
      cron: "0 2 * * *" # Example: daily at 02:00 UTC. Choose a time after warehouse loads.
9
      timezone: UTC
10
  # Runs every test of the connection right after sync_snowflake succeeds, so data
11
  # restated upstream shows as failing tests in the header bell. No schedule of its own.
12
  test_snowflake:
13
    command: kater test
14
    connection: snowflake_prod_connection
15
    deployment: production
16
    after: sync_snowflake
17
cache:
18
  results:
19
    fresh_for: 15m
20
  retention:
21
    unused_for: 7d
22
queries:
23
  warehouse_queries: 4

The file has three top-level keys, jobs, cache and queries. The cache and queries values here are the defaults, so the file works the same without them.

jobs

jobs maps each job’s name to what it runs and when. It is empty by default. Each key in the table is under the job’s name. Scheduled jobs shows how to set up the runner and run jobs by hand.

• Names. A job’s name, and the values of connection, deployment and after, use lowercase letters, digits and underscores. Each starts with a letter and has 64 characters at most.
• One trigger. Give each job a schedule or an after, not both.
• One job per command. Each command, connection and deployment has one job at most.
• Chains end in a schedule. after names a job in this file with the same deployment and connection. Kater refuses a loop of after jobs.
• Plain cron. Use five fields with no H or R, and put them in quotes. The expression must match a date: 0 0 31 2 * never does.
• Changed jobs are cancelled. A queued run whose job changed in the file is cancelled, with the code job_changed. So is a scheduled run whose schedule was turned off.

cache

cache sets how long Kater uses a query’s saved rows, and how long it keeps them. Kater reads it at each query run.

A duration is a whole number from 1, then one lowercase unit with no space. The units are s for seconds, m for minutes, h for hours and d for days. Examples are 90s, 15m, 6h and 30d. The longest is ten years, 3650d. Kater refuses a plain number, such as 900.

• Each person has their own rows. Row access can differ, so Kater keeps each person’s rows apart.
• Fresh rows skip the warehouse. A run within fresh_for gets the saved rows.
• Stale rows show at once. After fresh_for, a run shows the saved rows, and Kater runs the query again for new rows.
• Date windows end at midnight. Rows for a relative window, such as the last 7 days, turn stale at the next midnight. This happens even within fresh_for.
• A change applies to saved rows too. A shorter fresh_for makes older rows stale at once.
• Storage has its own limit. Kater can remove rows before unused_for when the server’s result storage is full. It keeps rows that are still in use.

queries

queries sets how many of your organisation’s queries Kater runs on the warehouse at once. Kater reads it at each query run.

• Size it to your warehouse. Run kater admin connection test <connection>. Its last line says how many queries the warehouse runs at once, such as “This warehouse runs 8 queries at once, on 1 cluster. Kater runs at most 4.” The connection’s page in the portal shows the same line under Last test.
• Leave room for other users. Other tools and people use the same warehouse. A value above the warehouse’s own limit makes queries wait in the warehouse, not in Kater.
• A change applies at the next query. Queries that already run are not stopped.
• The count is for your whole organisation. Kater runs on more than one server, and all of them share this count. Together they never run more than this many of your queries at once.
• Scheduled work uses at most half of the count. Work that nobody waits for, such as a workflow’s scheduled run and the nightly read of each figure’s weeks, runs at most half of queries.warehouse_queries at once, rounded down, and at least 1. People’s questions keep the rest. With the default of 4, scheduled work runs at most 2 queries at once, and people’s questions can always use 2. Scheduled work waits for its share, never for a person’s slot.

When every slot is busy

When many people run queries at the same time, a query waits in Kater’s queue. When the queue stays full for too long, the run fails with “Kater is running as many queries as it can right now. Run it again shortly.” In an app, the message says to reload the app.

When people see this often, and the warehouse has room, increase queries.warehouse_queries.

When the file is not valid

Kater checks the file each time it reads it. A query run checks only cache and queries, so a mistake in jobs stops only the jobs. A mistake in cache or queries stops both the query runs and the jobs. Kater never uses the defaults in place of a file it cannot read.

These make the file not valid:

• A key that this page does not list, at any level.
• A value of the wrong type. enabled: "true" is text, not true. fresh_for: 900 is a number, not a duration. warehouse_queries: "8" is text, not a number.
• A number out of its range, such as warehouse_queries: 0 or 33.
• A key two times in one mapping, such as two jobs with the same name.
• A job that breaks a rule in jobs.
• YAML that Kater cannot read, or a file larger than 128 KiB.

To find the problem, run kater job list in the content root on the server. It needs the same administrator identity as every kater job command. It names each problem by its place in the file:

kater job list --kater-root . --deployment production

{"error":"kater.config.yaml: jobs.sync_snowflake.schedule.cron: Cron expression has no matching date.","code":"invalid_request","diagnostics":[],"request_id":"ba623c3f1c768a99fbcbcbcad87f6381","error_reference":null,"warehouse":null}

Fix each problem and publish the file again. Query runs work again at once, with no restart.

Next steps

• Scheduled jobs
• Work in a local copy
• Files and access