Is Type Check Examples

The situation: A daily CSV lands in a staging table where every column is text. The amount_raw column must hold decimal numbers before the transformation casts it; rows holding sentinels break the cast and fail the whole load.

Check configuration

Payload

{
    "description": "Staging amount column must hold decimal numbers.",
    "rule": "isType",
    "fields": ["amount_raw"],
    "container_id": 145,
    "coverage": 1,
    "filter": null,
    "properties": {"field_type": "Fractional"},
    "tags": ["staging", "typing"],
    "additional_metadata": {"jira": "DATA-9201"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 12
}

Sample Data

What gets flagged

Row 2 holds the sentinel N/A, which cannot be read as a number. Row 3 is a negative whole number, which Fractional accepts. Row 4 is NULL and passes without firing an anomaly. Coverage is 100%, so the failing row is reported as a Record Anomaly.

Flowchart

graph TD
    A["No filter, evaluate all rows"] --> B["Read amount_raw"]
    B --> C{"Is value NULL?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E{"Can it be read<br/>as a Fractional?"}
    E -->|Yes| D
    E -->|No| F["Flag row.<br/>Record Anomaly per failing row."]

Equivalent SQL

-- Rows the Is Type check would flag.
SELECT s.*
FROM staging_amounts s
WHERE s.amount_raw IS NOT NULL
  AND TRY_CAST(s.amount_raw AS DOUBLE) IS NULL;

The situation: The events_raw table receives text from several producers. Only rows from the legacy producer are expected to carry a parseable date in event_date_raw; the other producers send an epoch value handled by a different rule.

Check configuration

Payload

{
    "description": "Legacy events must carry a parseable date.",
    "rule": "isType",
    "fields": ["event_date_raw"],
    "container_id": 512,
    "coverage": 1,
    "filter": "producer = 'legacy'",
    "properties": {"field_type": "Date"},
    "tags": ["events", "typing"],
    "additional_metadata": {"jira": "DATA-9244"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 29
}

Sample Data (filtered to producer = 'legacy')

Why the filter matters

The filter runs before the evaluation, so rows from producers that send epoch values are never tested as dates. Only legacy rows are evaluated.

What gets flagged

E-02 carries an annotated, ambiguous value that cannot be read as a date. Coverage is 100%, so the failure is reported as a Record Anomaly, and the message ends with the filter that scoped the evaluation.

Flowchart

graph TD
    A["Apply filter: producer = 'legacy'"] --> B["Read event_date_raw"]
    B --> C{"Can it be read<br/>as a Date?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E["Flag row.<br/>Anomaly message ends with<br/>[filter: producer = 'legacy']"]

Equivalent SQL

-- Rows the check would flag among legacy events.
SELECT e.*
FROM events_raw e
WHERE e.producer = 'legacy'
  AND e.event_date_raw IS NOT NULL
  AND TRY_CAST(e.event_date_raw AS DATE) IS NULL;

The situation: A legacy feed writes is_active_raw with a mix of textual booleans and free-form answers. A normalization job is rewriting them, and a small fraction is expected to remain unreadable until it finishes, so the check tolerates up to 0.5% failures.

Check configuration

Payload

{
    "description": "Active flag must hold a boolean value.",
    "rule": "isType",
    "fields": ["is_active_raw"],
    "container_id": 733,
    "coverage": 0.995,
    "filter": null,
    "properties": {"field_type": "Boolean"},
    "tags": ["legacy", "typing"],
    "additional_metadata": {"jira": "DATA-9288"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 41
}

Sample Data

What gets flagged

Records R-02 and R-03 hold free-form answers that cannot be read as booleans. R-04 is NULL, so it passes without firing an anomaly but still counts in the scanned total. Only 50% of the rows pass, which is below the 99.5% coverage threshold, so the check reports a single Shape Anomaly for the dataset with a sample of the offending rows. Coverage below 100% does not produce Record Anomalies.

Flowchart

graph TD
    A["No filter, evaluate all rows"] --> B["Read is_active_raw"]
    B --> C{"Is value NULL?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E{"Can it be read<br/>as a Boolean?"}
    E -->|Yes| D
    E -->|No| F["Flag row.<br/>Passing rate falls below the 99.5%<br/>coverage, so one Shape Anomaly is reported."]

Equivalent SQL

-- Rows the check would flag.
SELECT r.*
FROM legacy_records r
WHERE r.is_active_raw IS NOT NULL
  AND TRY_CAST(r.is_active_raw AS BOOLEAN) IS NULL;