Not Null Check FAQ

Answers to common questions about how the Not Null check tests for presence, how it handles empty strings and arrays, and how anomalies are reported, grouped by topic.

Behavior

Does an empty string count as null?

No. The rule tests for NULL only, so '', a space, and placeholders such as n/a all pass. Normalize blanks to NULL upstream when they should be treated as missing.

What happens when I select several fields?

They are asserted together: a row passes only when every selected field is populated, and fails as soon as one is empty. For the opposite rule, where one populated field is enough, use Any Not Null.

How are array fields evaluated?

By default the array is tested as a whole: a non-NULL array passes even if it is empty or holds NULL elements. Turn on Array Element Context to require every element to be populated.

Does the filter run before or after the evaluation?

Before. The platform applies the filter first and then evaluates only the rows that pass the filter. Filtered-out rows cannot trigger an anomaly and are not counted in the totals.

Anomaly Reporting

What do the anomaly messages look like?

Record Anomaly: <field> has no value

Shape Anomaly: <field> has no value in N of M asserted records

With two or three fields the names are joined with and, for example c_custkey and created_at have no value. From four fields on, the message counts them and lists the names at the end. See Anomaly Reporting for every form.

When a filter is set, the message names it at the end of its first sentence, as (applied to records where <expression>). Record and Shape Anomaly messages both carry it.

Does Not Null produce Record Anomalies, Shape Anomalies, or both?

At 100% coverage (the default), violating rows are reported as Record Anomalies. Below 100% coverage, a failed coverage assertion produces one Shape Anomaly for the dataset.

Scan settings can group a large number of Record Anomalies into one rolled-up Shape Anomaly, which carries the Shape Anomaly message. This behavior applies across rule types and is documented under Maximum Record Anomalies per Check.

Does Custom Anomaly Description work for Not Null?

Yes. Not Null emits Record Anomalies, so the anomaly_message_field payload field (and the Custom Anomaly Description toggle in the UI) applies.

When the option points at another column on the same row, the Record Anomaly message becomes that column's value for the violating row, shown as written. When the column is null, missing, or empty, the message falls back to the rule's standard wording, and when the column is masked the message reads <masked>. Shape Anomaly messages are unaffected.

Which cell is highlighted on a multi-field check?

Every selected field that is empty on the violating row. Fields outside the check render normally.

Configuration

Can I lower the coverage on a Not Null check?

Yes. Coverage at 1.0 (100%) means every row must pass. Lowering coverage to 0.995 allows up to 0.5% of rows to fail without raising an anomaly; once the failing fraction goes past that, the check reports a Shape Anomaly. Use lower coverage with care: a real regression that happens to fall just under the threshold will not raise an alert at all.

Can I add or remove fields on an existing check?

Yes. In the UI, open the check, change the Fields selection, and click Update; see Edit a Check for the full steps. Through the API, a PUT to /api/quality-checks/{id} updates fields. The rule type, the target container, and the associated Check Template stay immutable; see the API page for the editable/immutable matrix.

What permission do I need to create or edit a Not Null check?

The Drafter team permission on the datastore covers Draft work (creating a check as Draft or editing it while it stays Draft). Anything that puts the check into evaluation, such as creating or editing an Active check, archiving, or deleting, requires the Author team permission (or above). Viewing only requires Reporter. See Permissions for the full matrix.

Should I use one check for several columns or one per column?

Both work. One check is compact and easy to maintain; separate checks give each column its own anomaly stream, which is better when different teams own the fixes.