Volumetric Check Examples¶

The situation: An ETL job appends the day's orders to ORDERS every night and finishes by 23:30 UTC. When the job fails silently, the table keeps its old rows, so nothing looks wrong until a report is empty. The team wants an alert on any day that adds no rows.

Check configuration

Payload

{
    "description": "ORDERS must gain at least one row per daily load.",
    "rule": "volumetric",
    "fields": [],
    "container_id": 145,
    "filter": null,
    "properties": {"comparison": "Absolute Change", "window_size": 1, "min": 1, "max": null},
    "tags": ["daily-load"],
    "additional_metadata": {},
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 12
}

What it takes to alert

The check by itself only sets the expectation. Three more pieces turn it into an alert:

1. A Scan with the Metadata category runs every day at 00:10 UTC, after the load and before the hourly Observability measurement at 00:30. The Scan's count is then the only count of the day, so tomorrow's comparison sees exactly the rows this load left. See Schedule Options.
2. Volume Tracking is on for ORDERS, so a day without a Scan still gets a count and the history has no gap.
3. A Flow with the Anomaly Detected trigger, filtered by Check Rule Types = Volumetric and Anomaly's Tags = daily-load, sends the notification. Anomalies inherit the check's tags, which is what the tag filter matches. See Flow Triggers.

Sample Data

What gets flagged

Monday night's load fails, so Tuesday's Scan finds the same 25,000 rows that Monday's Scan counted. The check expects at least Monday's count plus 1, so the Scan reports a Shape Anomaly and the Flow sends the alert. On Wednesday the job catches up and the check passes again.

Flowchart

graph TD
    A["Scan counts the rows of ORDERS"] --> B["Read yesterday's count from the history"]
    B --> C{"Count today at least<br/>yesterday's count + 1?"}
    C -->|Yes| D["Check passes"]
    C -->|No| E["Report a Shape Anomaly"]
    E --> F["Flow matches rule type and tag<br/>and sends the alert"]

Equivalent SQL

-- The count the Scan compares against yesterday's count plus 1.
SELECT COUNT(*) FROM orders;

The situation: EVENTS grows with traffic, so its size drifts steadily, and weekends are quieter than weekdays. A fixed number of rows would be wrong most of the week. The team wants to know when the table is more than 20% smaller, or more than 50% larger, than it was on the same weekday last week.

Check configuration

Payload

{
    "description": "EVENTS stays within -20% and +50% of its size one week earlier.",
    "rule": "volumetric",
    "fields": [],
    "container_id": 212,
    "filter": null,
    "properties": {"comparison": "Percentage Change", "window_size": 7, "min": -0.2, "max": 0.5},
    "tags": ["traffic"],
    "additional_metadata": {},
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": null
}

Sample Data

What gets flagged

In week 2 a replayed backfill loaded the same events twice. The count more than doubled against the same Monday a week earlier, past the +50% limit. The days in between play no part: only the count from exactly seven days earlier is compared.

The expected maximum is the earlier count (1,180,000) plus 50%.

Flowchart

graph TD
    A["Scan counts the rows of EVENTS"] --> B["Read the count from 7 days earlier"]
    B --> C{"Change between -20%<br/>and +50%?"}
    C -->|Yes| D["Check passes"]
    C -->|No| E["Report a Shape Anomaly"]

The situation: CURRENCY_RATES is fully reloaded every day with one row per currency pair, so it should always hold about 160 rows. The team allows for a few pairs being added or retired, and wants to smooth out a single odd day.

Check configuration

Payload

{
    "description": "CURRENCY_RATES averages between 150 and 180 rows over three days.",
    "rule": "volumetric",
    "fields": [],
    "container_id": 98,
    "filter": null,
    "properties": {"comparison": "Absolute Value", "window_size": 3, "min": 150, "max": 180},
    "tags": ["reference-data"],
    "additional_metadata": {},
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": null
}

Sample Data

What gets flagged

On Wednesday a partial reload left 40 rows. The three-day average dropped to 121, below the Min of 150, so the Scan reports an anomaly. Because the low day stays inside the averaging period, Thursday and Friday also fail even though their own counts are normal. The check passes again on Saturday, once Wednesday leaves the period.

A longer Measurement Period smooths small swings, but it also keeps one bad day in the average for the whole period. Use 1 day when every day must stand on its own.

Flowchart

graph TD
    A["Scan counts the rows of CURRENCY_RATES"] --> B["Average with the previous two daily counts"]
    B --> C{"Average between<br/>150 and 180?"}
    C -->|Yes| D["Check passes"]
    C -->|No| E["Report a Shape Anomaly"]