Satisfies Expression Troubleshooting¶

Common problems when creating and running a Satisfies Expression check, their causes, and how to resolve them.

Validation Fails When Creating the Check¶

Clicking Validate returns an error instead of a success message.

Cause: The rule could not run against the selected data. The most common reasons are an invalid Expression, an invalid Filter Clause, or a container that failed to load.

Resolution:

1. Review the error message: it explains what needs attention.
2. Check the Filter Clause for typos: it must be a valid SQL WHERE expression for the selected container.
3. If the message says the container is marked as Unloadable, the container was skipped after repeated operation failures. Follow the steps in Unloadable Container Error.

Rows With an Empty Column Are Not Reported¶

The check passes records where a referenced column is empty, even though the condition does not hold for them.

Cause: A SQL comparison involving an empty value returns nothing rather than false, and a row is reported only when the expression is definitely false. Rows where it returns nothing pass.

Resolution: State the intent explicitly: col IS NOT NULL AND <condition> to report empty values, or col IS NULL OR <condition> to let them pass on purpose.

A CASE Statement Lets Rows That Match No Branch Pass¶

Records that none of the WHEN clauses cover are never flagged.

Cause: Without an ELSE, a CASE returns nothing for those rows, and a row that does not evaluate to false passes.

Resolution: Add an explicit ELSE FALSE to report those rows, or ELSE TRUE to make the pass explicit, so every row gets a definite verdict.

Validation Fails on the Expression¶

Clicking Validate returns an error instead of a success message.

Cause: The expression is not valid Spark SQL for this container: a misspelled column, an unbalanced parenthesis, a function the source does not provide, or an expression that returns a value instead of a condition.

Resolution: Read the error message, then check the column names against the container and confirm the expression evaluates to true or false rather than to a value.

Validation Fails With a Cast Error¶

Clicking Validate returns an error that starts with [CAST_INVALID_INPUT] and says a value cannot be cast to a type such as "DOUBLE" or "DATE".

Cause: A constant in the expression can never be converted to the type it is compared with, so the expression would fail on every row. The most common case is a number field compared with empty text: in PLY <> '' on a DOUBLE field, '' is not a number. A date field compared with text that is not a valid date, such as D = '2024-13-45', fails the same way, and a constant division by zero such as 1 / 0 fails with [DIVIDE_BY_ZERO]. Validate catches these before the check ever runs in a scan.

Resolution:

1. Compare the field with a value of its own type. For a number field, write PLY <> 0, or PLY IS NOT NULL when the intent is "has a value". For a date field, use a valid date such as D = '2024-12-31'.
2. When the value is text that may not convert, convert it in a way that tolerates bad input. TRY_CAST returns an empty value instead of failing, and COALESCE replaces that empty value with a default, for example COALESCE(TRY_CAST(amount_raw AS DOUBLE), 0) >= 0.

The Filter Clause is checked the same way: a filter such as PLY <> '' on a DOUBLE field is refused at Validate with the same error. Fix it the same way.

The Check Is Slow¶

The scan takes noticeably longer than other checks on the same container.

Cause: The expression runs once per row. A subquery inside it runs in that context too, which is far more expensive than a plain comparison.

Resolution: Add a Filter Clause so fewer rows reach the expression, simplify the condition, or move the heavy part into a Computed Field and check that instead.

The Anomaly Highlights Several Cells¶

The Source Records view marks more than one column on the violating record.

Cause: Every field on the check is highlighted, not only the value that made the row fail. The field list is derived from the expression, so by default that is every column the expression names. If the fields were selected by hand, the highlight follows that list instead.

Resolution: Read the expression quoted in the message against the highlighted values. Keeping expressions short makes this quick.

An Edited Check Keeps Behaving the Old Way¶

You changed the configuration but the anomaly list did not change.

Cause: Edits take effect on the next Scan. Saving the check does not re-evaluate the data, and anomalies raised under the previous configuration are not modified.

Resolution: Run a Scan on the container (or wait for the scheduled one). Old anomalies stay open until you triage them or a Full scan with Auto Resolve clears them. See What Happens to Existing Anomalies.

No Anomalies Although the Data Looks Wrong¶

A Scan ran, the data clearly breaks the rule, but nothing was reported.

Cause: One of these is excluding the violations:

• The check is in Draft status: Draft checks are not evaluated by Scans.
• The Filter Clause excludes those rows before the evaluation runs.
• Coverage is below 100% and the failing fraction stayed within the tolerance, so the check passed.
• A column the expression references is empty, or a CASE has no matching branch, so the expression returns nothing and the row passes. See Rows With an Empty Column Are Not Reported.

Resolution: Confirm the check is Active and test the filter expression against the offending rows. Then review the coverage setting; see Coverage and Tolerance.

Expected One Anomaly per Row, Got a Single Rolled-Up One¶

Many rows failed, but the scan reported one Shape Anomaly instead of per-row Record Anomalies.

Cause: When the number of failing rows exceeds the scan's rollup threshold, the per-row findings are grouped into one rolled-up Shape Anomaly.

Resolution: This is expected behavior; the rolled-up anomaly carries sampled source records. To change the threshold, see Maximum Record Anomalies per Check.