Not Empty Anomaly Reporting

How the Not Empty check reports violations on text and on array fields, and how the offending value appears in Source Records.

Anomaly Messages

What the check reports depends on the kind of field it runs on.

On a text field, violations are attributed to individual rows. At 100% coverage each row whose value is empty or made only of spaces produces its own Record Anomaly, naming the field and the offending record. Below 100% coverage, a failed coverage assertion produces one Shape Anomaly for the dataset instead, counting the rows that were blank.

On an array field, the check evaluates the field as a whole and reports a Shape Anomaly: rows are not called out individually, because the rule is asserting a property of the column rather than diagnosing each record.

In both cases the anomaly reports that the field does not hold content, so what you act on is the field and the rows behind it rather than a value that was out of range.

Record Anomaly

<field> is empty

Shape Anomaly

<field> is empty in N of M asserted records

With several fields on the check the verb agrees with them, for example <field_a> and <field_b> are empty. N is the number of rows with an empty value and M the number of rows the check asserted, after the filter if one is set.

When a filter is set, the message names it at the end of its first sentence, as (applied to records where <expression>). Record and Shape Anomaly messages both carry it.

Scan settings can group a large number of Record Anomalies into one rolled-up Shape Anomaly, which carries the Shape Anomaly message. This behavior applies across rule types and is documented under Maximum Record Anomalies per Check.

For a Shape Anomaly, the failing share appears in the opening line of the anomaly's description, for example 2 of 4 records (50%) in ACCOUNTS failed 1 check during full scan #4821. That line counts against every record the scan processed, so on a filtered check its denominator is no smaller than the check's own asserted count. See Anomaly Description.

Source Records Behavior

On a text field the offending cell is highlighted with an orange outline and an orange-tinted background in the Source Records view. An empty value and one made of spaces look identical on screen, so the highlight is what tells you which cell the check objected to.

On an array field the anomaly belongs to the column rather than to individual rows, so there is no per-row highlight to read.

Custom Anomaly Description

On a text field, Not Empty supports Custom Anomaly Description because it emits Record Anomalies. The option is anomaly_message_field in the API and the Custom Anomaly Description toggle in the UI.

When the option points at another column on the same row, the Record Anomaly message becomes that column's value for the violating row, shown as written. When the column is null, missing, or empty, the message falls back to the rule's standard wording, and when the column is masked the message reads <masked>. Shape Anomaly messages are unaffected.

Because the option only applies to Record Anomalies, it has no effect on Shape Anomalies, and therefore none on an array field.

See Also