Any Not Null Check Examples
Three real-world scenarios that show how the Any Not Null check is typically used in production: enforcing at least one customer contact channel, validating that alternative identifier fields are not all missing, and catching orders that lost every piece of business context after an ingestion regression. The first two run at 100% coverage and report Record Anomalies; the third lowers coverage and reports a Shape Anomaly instead.
The situation: BrightCart requires every customer to be reachable through at least one channel. The customers table exposes email, phone, and mobile. Any individual field can be blank, but a row with all three missing cannot be contacted and should be flagged.
Check configuration
| Field | Value |
|---|---|
| Rule | Any Not Null |
| Fields | email, phone, mobile |
| Filter | (none) |
| Coverage | 100% |
| Custom Anomaly Description | Off |
| Status | Active |
| Owner | (check creator) |
| Anomaly Assignee | (Customer Data on-call) |
| Tags | customers, completeness |
| Additional Metadata | jira: DATA-2145 |
| Description | Every customer must be reachable through at least one of email, phone, or mobile. |
Payload
{
"description": "Every customer must be reachable through at least one of email, phone, or mobile.",
"rule": "anyNotNull",
"fields": ["email", "phone", "mobile"],
"container_id": 145,
"coverage": 1,
"filter": null,
"properties": {},
"tags": ["customers", "completeness"],
"additional_metadata": {"jira": "DATA-2145"},
"anomaly_message_field": null,
"template_id": null,
"status": "Active",
"owner_id": 7,
"default_anomaly_assignee_id": 12
}
Sample Data
| customer_id | phone | mobile | |
|---|---|---|---|
| C001 | alice@example.com | (null) | (null) |
| C002 | (null) | +1 415 555 0110 | (null) |
| C003 | (null) | (null) | (null) |
| C004 | (null) | (null) | +1 415 555 0142 |
What gets flagged
Row C003 has NULL in every selected field, so it fails. The other rows carry at least one non-NULL contact and pass. Coverage is 100%, so the failure is reported as a Record Anomaly on that row.
Record Anomaly
None of the fields 'email, phone, mobile' have a value assigned
Flowchart
graph TD
A["No filter, evaluate all rows"] --> B["Read email, phone, mobile"]
B --> C{"Is any of the three non-NULL?"}
C -->|Yes| D["Row passes"]
C -->|No| E["Flag row.<br/>Coverage is 100%, so each failing<br/>row becomes a Record Anomaly."]
Equivalent SQL
The situation: An identity onboarding pipeline lands rows in the applicants table. Applicants coming from the residents track (applicant_type = 'resident') must carry at least one of national_id, passport_number, or driver_license. Non-resident rows follow a different flow and should not be evaluated by this check.
Check configuration
| Field | Value |
|---|---|
| Rule | Any Not Null |
| Fields | national_id, passport_number, driver_license |
| Filter | applicant_type = 'resident' |
| Coverage | 100% |
| Custom Anomaly Description | Off |
| Status | Active |
| Owner | (check creator) |
| Anomaly Assignee | (Identity Platform on-call) |
| Tags | identity, onboarding |
| Additional Metadata | jira: DATA-3081 |
| Description | Resident applicants must supply at least one government-issued identifier. |
Payload
{
"description": "Resident applicants must supply at least one government-issued identifier.",
"rule": "anyNotNull",
"fields": ["national_id", "passport_number", "driver_license"],
"container_id": 212,
"coverage": 1,
"filter": "applicant_type = 'resident'",
"properties": {},
"tags": ["identity", "onboarding"],
"additional_metadata": {"jira": "DATA-3081"},
"anomaly_message_field": null,
"template_id": null,
"status": "Active",
"owner_id": 7,
"default_anomaly_assignee_id": 18
}
Sample Data (filtered to applicant_type = 'resident')
| applicant_id | applicant_type | national_id | passport_number | driver_license |
|---|---|---|---|---|
| A-001 | resident | 111-22-3333 | (null) | (null) |
| A-002 | resident | (null) | X99887766 | (null) |
| A-003 | resident | (null) | (null) | (null) |
Why the filter matters
The filter runs before the check. Non-resident applicants are ignored entirely, so their identifiers (or lack of them) never contribute to the anomaly counts.
What gets flagged
Applicant A-003 has all three identifier fields NULL and is the only row reported. Coverage is 100%, so the failure is reported as a Record Anomaly, and the message ends with the filter that scoped the evaluation.
Record Anomaly
None of the fields 'national_id, passport_number, driver_license' have a value assigned [filter: applicant_type = 'resident']
Flowchart
graph TD
A["Apply filter: applicant_type = 'resident'"] --> B["Read national_id, passport_number, driver_license"]
B --> C{"Is any of the three non-NULL?"}
C -->|Yes| D["Row passes"]
C -->|No| E["Flag row.<br/>Anomaly message ends with<br/>[filter: applicant_type = 'resident']"]
Equivalent SQL
The situation: SunriseMart records every order in the orders table. Each row is expected to carry at least one of o_comment (customer or system notes) or o_orderstatus (state such as Pending, Shipped, or Cancelled). Either field on its own is enough context; a row with both fields NULL is unusable by fulfillment and support. A small backfill is planned, so the team allows up to 0.5% of rows to fail before a Shape Anomaly is fired.
Check configuration
| Field | Value |
|---|---|
| Rule | Any Not Null |
| Fields | o_comment, o_orderstatus |
| Filter | (none) |
| Coverage | 99.5% |
| Custom Anomaly Description | Off |
| Status | Active |
| Owner | (check creator) |
| Anomaly Assignee | (Order Operations) |
| Tags | orders, completeness |
| Additional Metadata | jira: DATA-4102 |
| Description | Every order must carry either a comment or a status. |
Payload
{
"description": "Every order must carry either a comment or a status.",
"rule": "anyNotNull",
"fields": ["o_comment", "o_orderstatus"],
"container_id": 318,
"coverage": 0.995,
"filter": null,
"properties": {},
"tags": ["orders", "completeness"],
"additional_metadata": {"jira": "DATA-4102"},
"anomaly_message_field": null,
"template_id": null,
"status": "Active",
"owner_id": 7,
"default_anomaly_assignee_id": 24
}
Sample Data
| o_orderkey | o_comment | o_orderstatus |
|---|---|---|
| 1 | (null) | (null) |
| 2 | Good product | (null) |
| 3 | (null) | Shipped |
| 4 | Rush ship | Pending |
What gets flagged
Order 1 fails because both o_comment and o_orderstatus are NULL. Orders 2, 3, and 4 each carry at least one non-NULL field and pass. Only 75% of the rows pass, which is below the 99.5% coverage threshold, so the check reports a single Shape Anomaly for the dataset with a sample of the offending rows. Coverage below 100% does not produce Record Anomalies.
Shape Anomaly
For the fields 'o_comment, o_orderstatus', 25.000% of 4 records (1) have no value set for any field
Flowchart
graph TD
A["No filter, evaluate all rows"] --> B["Read o_comment, o_orderstatus"]
B --> C{"Is either field non-NULL?"}
C -->|Yes| D["Row passes"]
C -->|No| E["Flag row.<br/>Passing rate falls below the 99.5%<br/>coverage, so one Shape Anomaly is reported."]
Equivalent SQL
Related
- Introduction: formal definition, field scope, and general/anomaly properties.
- How It Works: full semantics, NULL handling, filter behavior, and edge cases.
- API: payload shape and field notes for creating an Any Not Null check programmatically.
- FAQ: short answers to the most frequent questions.