Skip to content

Any Not Null Check Examples

Three real-world scenarios that show how the Any Not Null check is typically used in production: enforcing at least one customer contact channel, validating that alternative identifier fields are not all missing, and catching orders that lost every piece of business context after an ingestion regression. The first two run at 100% coverage and report Record Anomalies; the third lowers coverage and reports a Shape Anomaly instead.

The situation: BrightCart requires every customer to be reachable through at least one channel. The customers table exposes email, phone, and mobile. Any individual field can be blank, but a row with all three missing cannot be contacted and should be flagged.

Check configuration

Field Value
Rule Any Not Null
Fields email, phone, mobile
Filter (none)
Coverage 100%
Custom Anomaly Description Off
Status Active
Owner (check creator)
Anomaly Assignee (Customer Data on-call)
Tags customers, completeness
Additional Metadata jira: DATA-2145
Description Every customer must be reachable through at least one of email, phone, or mobile.

Payload

{
    "description": "Every customer must be reachable through at least one of email, phone, or mobile.",
    "rule": "anyNotNull",
    "fields": ["email", "phone", "mobile"],
    "container_id": 145,
    "coverage": 1,
    "filter": null,
    "properties": {},
    "tags": ["customers", "completeness"],
    "additional_metadata": {"jira": "DATA-2145"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 12
}

Sample Data

customer_id email phone mobile
C001 alice@example.com (null) (null)
C002 (null) +1 415 555 0110 (null)
C003 (null) (null) (null)
C004 (null) (null) +1 415 555 0142

What gets flagged

Row C003 has NULL in every selected field, so it fails. The other rows carry at least one non-NULL contact and pass. Coverage is 100%, so the failure is reported as a Record Anomaly on that row.

Record Anomaly

None of the fields 'email, phone, mobile' have a value assigned

Flowchart

graph TD
    A["No filter, evaluate all rows"] --> B["Read email, phone, mobile"]
    B --> C{"Is any of the three non-NULL?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E["Flag row.<br/>Coverage is 100%, so each failing<br/>row becomes a Record Anomaly."]

Equivalent SQL

-- Rows the Any Not Null check would flag.
SELECT c.*
FROM customers c
WHERE c.email IS NULL
  AND c.phone IS NULL
  AND c.mobile IS NULL;

The situation: An identity onboarding pipeline lands rows in the applicants table. Applicants coming from the residents track (applicant_type = 'resident') must carry at least one of national_id, passport_number, or driver_license. Non-resident rows follow a different flow and should not be evaluated by this check.

Check configuration

Field Value
Rule Any Not Null
Fields national_id, passport_number, driver_license
Filter applicant_type = 'resident'
Coverage 100%
Custom Anomaly Description Off
Status Active
Owner (check creator)
Anomaly Assignee (Identity Platform on-call)
Tags identity, onboarding
Additional Metadata jira: DATA-3081
Description Resident applicants must supply at least one government-issued identifier.

Payload

{
    "description": "Resident applicants must supply at least one government-issued identifier.",
    "rule": "anyNotNull",
    "fields": ["national_id", "passport_number", "driver_license"],
    "container_id": 212,
    "coverage": 1,
    "filter": "applicant_type = 'resident'",
    "properties": {},
    "tags": ["identity", "onboarding"],
    "additional_metadata": {"jira": "DATA-3081"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 18
}

Sample Data (filtered to applicant_type = 'resident')

applicant_id applicant_type national_id passport_number driver_license
A-001 resident 111-22-3333 (null) (null)
A-002 resident (null) X99887766 (null)
A-003 resident (null) (null) (null)

Why the filter matters

The filter runs before the check. Non-resident applicants are ignored entirely, so their identifiers (or lack of them) never contribute to the anomaly counts.

What gets flagged

Applicant A-003 has all three identifier fields NULL and is the only row reported. Coverage is 100%, so the failure is reported as a Record Anomaly, and the message ends with the filter that scoped the evaluation.

Record Anomaly

None of the fields 'national_id, passport_number, driver_license' have a value assigned [filter: applicant_type = 'resident']

Flowchart

graph TD
    A["Apply filter: applicant_type = 'resident'"] --> B["Read national_id, passport_number, driver_license"]
    B --> C{"Is any of the three non-NULL?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E["Flag row.<br/>Anomaly message ends with<br/>[filter: applicant_type = 'resident']"]

Equivalent SQL

-- Rows the check would flag among resident applicants.
SELECT a.*
FROM applicants a
WHERE a.applicant_type = 'resident'
  AND a.national_id IS NULL
  AND a.passport_number IS NULL
  AND a.driver_license IS NULL;

The situation: SunriseMart records every order in the orders table. Each row is expected to carry at least one of o_comment (customer or system notes) or o_orderstatus (state such as Pending, Shipped, or Cancelled). Either field on its own is enough context; a row with both fields NULL is unusable by fulfillment and support. A small backfill is planned, so the team allows up to 0.5% of rows to fail before a Shape Anomaly is fired.

Check configuration

Field Value
Rule Any Not Null
Fields o_comment, o_orderstatus
Filter (none)
Coverage 99.5%
Custom Anomaly Description Off
Status Active
Owner (check creator)
Anomaly Assignee (Order Operations)
Tags orders, completeness
Additional Metadata jira: DATA-4102
Description Every order must carry either a comment or a status.

Payload

{
    "description": "Every order must carry either a comment or a status.",
    "rule": "anyNotNull",
    "fields": ["o_comment", "o_orderstatus"],
    "container_id": 318,
    "coverage": 0.995,
    "filter": null,
    "properties": {},
    "tags": ["orders", "completeness"],
    "additional_metadata": {"jira": "DATA-4102"},
    "anomaly_message_field": null,
    "template_id": null,
    "status": "Active",
    "owner_id": 7,
    "default_anomaly_assignee_id": 24
}

Sample Data

o_orderkey o_comment o_orderstatus
1 (null) (null)
2 Good product (null)
3 (null) Shipped
4 Rush ship Pending

What gets flagged

Order 1 fails because both o_comment and o_orderstatus are NULL. Orders 2, 3, and 4 each carry at least one non-NULL field and pass. Only 75% of the rows pass, which is below the 99.5% coverage threshold, so the check reports a single Shape Anomaly for the dataset with a sample of the offending rows. Coverage below 100% does not produce Record Anomalies.

Shape Anomaly

For the fields 'o_comment, o_orderstatus', 25.000% of 4 records (1) have no value set for any field

Flowchart

graph TD
    A["No filter, evaluate all rows"] --> B["Read o_comment, o_orderstatus"]
    B --> C{"Is either field non-NULL?"}
    C -->|Yes| D["Row passes"]
    C -->|No| E["Flag row.<br/>Passing rate falls below the 99.5%<br/>coverage, so one Shape Anomaly is reported."]

Equivalent SQL

-- Rows the check would flag.
SELECT o.*
FROM orders o
WHERE o.o_comment IS NULL
  AND o.o_orderstatus IS NULL;
  • Introduction: formal definition, field scope, and general/anomaly properties.
  • How It Works: full semantics, NULL handling, filter behavior, and edge cases.
  • API: payload shape and field notes for creating an Any Not Null check programmatically.
  • FAQ: short answers to the most frequent questions.