Skip to content

Expected Values Check FAQ

Answers to common questions about how the Expected Values check evaluates values, how it handles NULLs and types, and how anomalies are reported, grouped by topic.

Behavior

How are NULLs treated?

NULL values pass the check. The engine evaluates column IS NULL OR column IN (list), so a row with a NULL in the selected field is never reported as a violation. If missing values are also a problem, pair Expected Values with a Not Null check on the same field.

Is matching case-sensitive?

Yes. "O" does not match "o". The comparison is an exact string match. If you need case-insensitive matching, normalize the data upstream or use a Satisfies Expression check with lower(field) IN ('o', 'f', 'p').

Is whitespace trimmed?

No. " O " does not match "O". The Authored Check modal shows a warning when an entry has leading or trailing whitespace, but it does not trim the entry for you. Clean the list before saving the check, or normalize the column upstream.

Does the filter clause run before or after the comparison?

Before. The platform applies the filter first and then evaluates the value comparison only on the rows that pass the filter. Rows outside the filter cannot trigger an Expected Values anomaly.

What happens when the list type does not match the field type?

The platform cannot interpret string entries as numbers or booleans, so every non-NULL row in the column fails the check. For example, an Expected Values check on an integer column with a string list ["1", "2", "3"] flags every row. Always match the list type to the field type: numeric lists for numeric fields, boolean lists for boolean fields, string lists for string/date/timestamp fields.

Array Fields

Can Expected Values evaluate every element of an array?

Yes, for Array[String] fields. The platform switches to element-wise evaluation automatically when the target field is an Array[String]: every element must be in the list, and a single out-of-vocabulary element invalidates the row.

Element-wise evaluation is supported only for string element lists today. For other array element types, the check falls back to scalar comparison and is not currently recommended.

Do empty or NULL arrays pass?

Yes. An empty array ([]) has no elements to evaluate and passes by definition. A NULL array also passes (the standard NULL handling rule applies). If you want to flag empty arrays as a violation, pair Expected Values with a separate check (such as a Satisfies Expression on size(field) > 0).

Anomaly Reporting

Does Expected Values produce Record or Shape Anomalies?

Both. On a scalar field it is controlled by coverage:

  • Coverage = 1 (100%, default): every failing row emits a Record Anomaly with the offending value in the message.
  • Coverage < 1: failures are rolled up into a single Shape Anomaly for the dataset, fired only when the failing fraction exceeds the threshold.

On an Array field the rule is evaluated element-wise as a field-level check, which always reports a Shape Anomaly, whatever the coverage. See Array Fields.

Scan settings can group a large number of Record Anomalies into one rolled-up Shape Anomaly, which carries the Shape Anomaly message. This behavior applies across rule types and is documented under Maximum Record Anomalies per Check.

What do the anomaly messages look like?

Record Anomaly: <field> has the value <value>, which is not among the allowed values: <allowed values>

Shape Anomaly: <field> values are not among those allowed, affecting N of M asserted records: <allowed values>

N is the number of rows whose value was non-NULL and not in the list, and M the number of rows the check asserted. See Anomaly Reporting for what each part means.

When a filter is set, the message names it at the end of its first sentence, as (applied to records where <expression>). Record and Shape Anomaly messages both carry it.

How are source records highlighted in the app?

The platform highlights only the offending cell in the field the check applies to. Other columns in the same row are rendered normally. This matches the rule's semantics: the violation is about a single field's value, not the entire row. The Sample Data tables in Examples mirror this convention.

Configuration

Can I paste a list of values from a spreadsheet?

Yes. The List field in the Authored Check modal accepts pasted values; the platform splits the pasted text on line breaks, trims leading and trailing whitespace from each line, and drops empty lines. This is convenient for importing a vocabulary maintained in a spreadsheet column.

Can I use Expected Values together with a Check Template?

Yes. Set template_id to the ID of an existing Check Template. The template's properties (including the list) are reused by the check, and any future updates to the template propagate to every check linked to it.

Does Custom Anomaly Description work for Expected Values?

Yes, for Record Anomalies. The anomaly_message_field payload field (and the Custom Anomaly Description toggle in the UI) applies.

When the option points at another column on the same row, the Record Anomaly message becomes that column's value for the violating row, shown as written. When the column is null, missing, or empty, the message falls back to the rule's standard wording, and when the column is masked the message reads <masked>. Shape Anomaly messages are unaffected.

What permission do I need to create or edit an Expected Values check?

The Drafter team permission on the datastore covers Draft work (creating a check as Draft or editing it while it stays Draft). Anything that puts the check into evaluation, such as creating or editing an Active check, archiving, or deleting, requires the Author team permission (or above). Viewing only requires Reporter. See Permissions for the full matrix.