Greater Than Field Best Practices

Guidelines for getting reliable signal from Greater Than Field checks while keeping the noise (and the maintenance) low.

Decide what an equal pair means

Two dates on the same day, or two amounts that match exactly, are common in real data. The Inclusive setting decides whether those rows pass, and getting it wrong produces either a flood of anomalies or a blind spot at exactly the boundary that matters.

Compare against a column, not a snapshot of it

The point of Greater Than Field is that the reference moves with the data. When you find yourself tempted to hardcode today's value of the other column, use Greater Than instead and be explicit that the boundary is a constant.

Set a tolerance only for noise you can name

Two timestamps written by different systems, or two amounts that accumulate rounding, differ for reasons that have nothing to do with data quality. Those are worth a comparator. A margin added to silence an inconvenient anomaly hides the defect the check exists to catch.

Remember NULLs make the row pass

A row where either side is missing is never reported, so a broken upstream job that nulls a column also silences this check. Pair it with Not Null on both fields when their presence is part of the rule.

Pair with Not Null when the value is mandatory

NULL values pass. The rule asserts that present values satisfy it; it does not require the field to be populated. When the field must also have a value, add a Not Null check on the same field.

Keep coverage at 100% unless a known backlog exists

At 100% coverage every failing row is reported as a Record Anomaly, which tells you exactly which values break the rule. Below 100% the check reports a single Shape Anomaly only when the failing fraction crosses the tolerance, and no per-row detail is produced. Lower coverage only while a known set of legacy rows is being cleaned up.

Scope with a filter instead of loosening coverage

When the rule only applies to part of the table (one product line, one tier, one period), express that with a filter clause rather than by lowering coverage. The filter removes the out-of-scope rows from evaluation entirely, and the expression is echoed in every anomaly message.

Route the anomalies to the right people

An out-of-bounds number usually comes from an application that failed to validate its input, or from a unit mismatch upstream. Set an Anomaly Assignee from the team that owns that producer, and tag the check so related checks are easy to find.

See Also