Not Exists In Check FAQ

Answers to common questions about how the Not Exists In check looks values up against another container, how NULLs and the two filters behave, and how anomalies are reported, grouped by topic.

Behavior

Does an empty reference container make the check pass?

Yes. With nothing in the blocked set, no value can be found in it, so every row passes. That is worth remembering when a reference filter is so narrow it returns nothing: the check goes quiet rather than strict.

Can the reference live in another datastore?

Yes. Set the reference Datastore on the Right Reference panel. Leave it empty when the reference container is in the same datastore as the target.

What is the difference between the two filter clauses?

The one on the main form scopes the target rows, deciding what gets evaluated. The one on the Right Reference panel scopes the reference rows, deciding which values are valid.

How are NULL values treated?

A NULL in the target field passes: there is no value to look up. Pair the check with Not Null when the value must also exist.

Can the target and the reference be the same container?

Yes. Point the reference at the same container and pick the field that forms the valid set. That is a common way to assert an internal relationship.

Anomaly Reporting

What do the anomaly messages look like?

Record Anomaly: The field '<field_name>' has value '<row_value>', which unexpectedly exists in '<reference_field>'

Shape Anomaly: For the field '<field_name>', X.XXX% of N records (K) have values that unexpectedly exist in '<reference_field>'

When a filter is set, both messages end with [filter: <expression>].

Does Not Exists In produce Record Anomalies, Shape Anomalies, or both?

At 100% coverage (the default), violating rows are reported as Record Anomalies. Below 100% coverage, a failed coverage assertion produces one Shape Anomaly for the dataset. A scan can also roll up a large number of Record Anomalies into one Shape Anomaly.

Does Custom Anomaly Description work for Not Exists In?

Yes. Not Exists In emits Record Anomalies, so the anomaly_message_field payload field (and the Custom Anomaly Description toggle in the UI) replaces the Record Anomaly message with the value of the named column on the violating row. When that column is null, missing, or empty, the standard Record template is used instead. The Shape Anomaly always uses the fixed template.

Configuration

Can I lower the coverage on a Not Exists In check?

Yes. Coverage at 1.0 (100%) means every row must pass. Lowering coverage to 0.995 allows up to 0.5% of rows to fail without raising an anomaly; once the failing fraction goes past that, the check reports a Shape Anomaly. Use lower coverage with care: a real regression that happens to fall just under the threshold will not raise an alert at all.

Can I change the reference on an existing check?

Yes. In the UI, open the check, change the Right Reference panel, and click Update; see Edit a Check for the full steps. Through the API, a PUT to /api/quality-checks/{id} updates the properties object. The rule type, the target container, and the associated Check Template stay immutable.

What permission do I need to create or edit a Not Exists In check?

The Drafter team permission on the datastore covers Draft work (creating a check as Draft or editing it while it stays Draft). Anything that puts the check into evaluation, such as creating or editing an Active check, archiving, or deleting, requires the Author team permission (or above). Viewing only requires Reporter. See Permissions for the full matrix.

How is Not Exists In different from Exists In?

They are mirror images. Not Exists In requires the value to be absent from the reference set; Exists In requires it to be present in it.