How Greater Than Checks Work

Definition

Asserts that every value in a numeric field is greater than a configured threshold.

Overview

The Greater Than rule compares a single numeric field against a fixed threshold. Each row passes when its value is greater than the configured Value. The Inclusive setting decides whether a value exactly equal to the threshold passes (>=) or fails (>). An optional Numeric comparator adds a tolerance, so values that miss the threshold by less than the configured margin are still accepted.

Typical use cases:

• Enforce a floor on an amount, a score, or a duration, deciding whether the boundary itself qualifies.
• Keep a reconciled measure above a limit while tolerating the rounding a pipeline introduces.
• Express an eligibility rule such as a minimum tenure or a minimum order value.

Field Scope

Single: The rule evaluates exactly one field per check.

Accepted Types

General Properties

The filter allows you to define a subset of data upon which the rule will operate.

It requires a valid Spark SQL expression that determines the criteria rows in the DataFrame should meet. This means the expression specifies which rows the DataFrame should include based on those criteria. Since it's applied directly to the Spark DataFrame, traditional SQL constructs like WHERE clauses are not supported.

O_TOTALPRICE > 1000

C_MKTSEGMENT = 'BUILDING'

WHERE O_TOTALPRICE > 1000

WHERE C_MKTSEGMENT = 'BUILDING'

O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

(L_SHIPDATE = '1998-09-02' OR L_RECEIPTDATE = '1998-09-01') AND L_RETURNFLAG = 'R'

WHERE O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

O_TOTALPRICE > 1000, O_ORDERSTATUS = 'O'

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - INSTR('-', O_ORDERPRIORITY)
) = 'URGENT'

LEVENSHTEIN(C_NAME, 'Supplier#000000001') < 7

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - CHARINDEX('-', O_ORDERPRIORITY)
) = 'URGENT'

EDITDISTANCE(C_NAME, 'Supplier#000000001') < 7

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM {{_qualytics_self}}
    WHERE O_TOTALPRICE > 1000
)

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM ORDERS
    WHERE O_TOTALPRICE > 1000
)

Specific Properties

Greater Than has three rule-specific properties:

Anomaly Types

Evaluation Flow

Every Greater Than check follows the same four-step evaluation flow:

1. Apply the filter clause. If the check has a filter set, only rows matching the filter expression continue to the next step. Rows outside the filter are ignored and cannot contribute to the violation count.
2. Read the field value. The platform reads the numeric value for the current row.
3. Compare against the threshold. With no tolerance configured, the row passes when the value is greater than Value, using >= or > depending on the Inclusive setting. With a tolerance, the margin is added to the value before the comparison runs.
4. Apply coverage. At 100% coverage, any failing row causes the check to fail. Below 100% coverage, the check fails only when the passing fraction drops below the threshold (see Coverage and Tolerance).

Inclusivity

The Inclusive setting changes what happens exactly at the threshold. With it on, the comparison is >= and a value equal to Value passes. With it off, the comparison is > and that same value is flagged. When the threshold is a business boundary that people argue about, spell the decision out in the check's description so the setting is not silently flipped later.

The Numeric Comparator

The optional Numeric comparator gives the comparison a tolerance. The margin is added to the row's value before the threshold is applied, so a value that misses by less than the margin still passes. The margin can be absolute (a fixed amount) or relative (a percentage of the value being compared), which matters when the magnitudes in the column vary widely.

Use it for measures where tiny deviations are noise rather than defects, such as amounts that accumulate rounding through a pipeline. Leave it empty when the threshold is a hard limit.

NULL Handling

The check passes NULL values: a row with NULL in the evaluated field is not counted as a violation. Greater Than only asserts that present values satisfy the comparison.

If the field must also be populated, pair it with a Not Null check on the same field.

Numeric Fields Inside an Array

An array column itself cannot be selected: the rule accepts Integral and Fractional only, and anything else is rejected with 422. A numeric field nested inside an array of structs can be selected, and there the rule is evaluated element-wise: every element is compared against the threshold and the row fails as soon as one element fails. Empty arrays and NULL arrays pass, since there is nothing to evaluate.

Element-wise evaluation runs as a field-level check, so it reports a Shape Anomaly for the field instead of per-row Record Anomalies, whatever the coverage is set to.

The Filter Clause

The filter clause is a SQL WHERE expression applied before the evaluation. Filtered-out rows are ignored entirely (they cannot trigger a violation and are not counted in the totals).

When a filter is set, both the Record Anomaly and the Shape Anomaly messages end with [filter: <expression>] so the evaluated scope is visible in the alert.

Coverage and Tolerance

Coverage is a fractional value between 0 and 1 that defines the minimum fraction of evaluated rows that must pass:

• 1.0 (100%, default): every row in the filtered set must pass. Any failing row causes the check to fail. This is the strictest setting.
• < 1.0: the check tolerates a fraction of rows failing. The check fails only when the fraction of passing rows drops below the threshold.

Lower coverage values are useful when a small, known fraction of failures is expected. Use coverage carefully: a 0.5% tolerance can mask a real regression that happens to fall just under the threshold.

See Also