How After Date Time Checks Work

Definition

Asserts that every value in a Date or Timestamp field is strictly later than a chosen cutoff date and time.

Overview

The After Date Time rule defines a lower time boundary on a single date or timestamp field. Each row in the target container must hold a value that is greater than the cutoff; any value equal to or earlier than the cutoff fires an anomaly. The comparison is strict (>), not inclusive (>=), and the cutoff is stored as a UTC instant.

Typical use cases:

• Enforce a system go-live or migration cutoff.
• Validate ingestion or processing timestamps against a freshness floor.
• Detect stale, replayed, or backfilled rows that slipped through a date-scoped pipeline.

Field Scope

Single: The rule evaluates exactly one field per check.

Accepted Types

General Properties

The filter allows you to define a subset of data upon which the rule will operate.

It requires a valid Spark SQL expression that determines the criteria rows in the DataFrame should meet. This means the expression specifies which rows the DataFrame should include based on those criteria. Since it's applied directly to the Spark DataFrame, traditional SQL constructs like WHERE clauses are not supported.

O_TOTALPRICE > 1000

C_MKTSEGMENT = 'BUILDING'

WHERE O_TOTALPRICE > 1000

WHERE C_MKTSEGMENT = 'BUILDING'

O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

(L_SHIPDATE = '1998-09-02' OR L_RECEIPTDATE = '1998-09-01') AND L_RETURNFLAG = 'R'

WHERE O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

O_TOTALPRICE > 1000, O_ORDERSTATUS = 'O'

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - INSTR('-', O_ORDERPRIORITY)
) = 'URGENT'

LEVENSHTEIN(C_NAME, 'Supplier#000000001') < 7

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - CHARINDEX('-', O_ORDERPRIORITY)
) = 'URGENT'

EDITDISTANCE(C_NAME, 'Supplier#000000001') < 7

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM {{_qualytics_self}}
    WHERE O_TOTALPRICE > 1000
)

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM ORDERS
    WHERE O_TOTALPRICE > 1000
)

Specific Properties

After Date Time has one rule-specific property:

Anomaly Types

Evaluation Flow

Every After Date Time check follows the same four-step evaluation flow:

1. Apply the filter clause. If the check has a filter set, only rows matching the filter expression continue to the next step. Rows outside the filter are ignored and cannot contribute to the violation count.
2. Read the field value as a timestamp. The platform interprets each row's value as a timestamp for comparison; values that cannot be interpreted are treated as failing the comparison (see below).
3. Compare against the cutoff. The cutoff (stored as a UTC instant) is compared to the field value using strict greater-than (>). A row passes only when the field value is strictly later than the cutoff.
4. Apply coverage. At 100% coverage, any failing row causes the check to fail. Below 100% coverage, the check fails only when the passing fraction drops below the threshold (see Coverage and Tolerance).

The comparison is strict, so a row whose field value equals the cutoff is flagged. To get inclusive semantics, set the cutoff earlier than the boundary you want to allow. The form's date and time picker goes down to the minute, so the closest setting there is the preceding minute, which also lets through any value inside that minute. Through the API the cutoff accepts an exact instant, down to fractions of a second, so the window can be made as narrow as the data needs. On a Date field, one day earlier is the natural step.

NULL Handling

The check passes NULL values: a row with NULL in the evaluated field is not counted as a violation. This is intentional. After Date Time only asserts that present values fall after the cutoff and does not enforce mandatory presence on the field.

If the field must also be populated, pair After Date Time with a Not Null check on the same field. Together they enforce "the field must be present AND must be later than the cutoff".

Values that cannot be interpreted as a timestamp are flagged as violations. Because the field picker accepts only Date and Timestamp columns, this rarely happens in practice.

Cutoff and Time Zone Behavior

The cutoff is stored as a UTC instant. When you set the cutoff to 2025-12-01 06:15:00 in the UI, the platform records it as 2025-12-01T06:15:00Z and uses that exact instant for every evaluation.

The platform evaluates the comparison in UTC:

• Timestamp fields stored with explicit time zone offsets are compared directly against the UTC cutoff.
• Timestamp fields without an explicit time zone are interpreted as UTC instants before the comparison.
• Date fields are interpreted as midnight UTC of that calendar day, then compared to the cutoff instant.

If your source data carries values in mixed or local time zones, normalize the field upstream with a Computed Field before running After Date Time so the comparison is unambiguous.

The Filter Clause

The filter clause is a SQL WHERE expression applied before the evaluation. Filtered-out rows are ignored entirely (they cannot trigger a violation and are not counted in the totals).

Common uses:

• Restricting the boundary to a subset of the dataset (tenant_id = 42, status = 'active').
• Excluding known-bad legacy rows that are tracked by a separate clean-up task.
• Scoping the check to a partition (event_date >= '2025-12-01').

When a filter is set, both the Record Anomaly and the Shape Anomaly messages end with [filter: <expression>] so the evaluated scope is visible in the alert.

Coverage and Tolerance

Coverage is a fractional value between 0 and 1 that defines the minimum fraction of evaluated rows that must pass:

• 1.0 (100%, default): every row in the filtered set must pass. Any failing row causes the check to fail. This is the strictest setting.
• < 1.0: the check tolerates a fraction of rows failing the comparison. The check fails only when the fraction of passing rows drops below the threshold.

Lower coverage values are useful when a small, known fraction of pre-cutover rows is expected (for example, during a slow migration). Use coverage carefully: a 0.5% tolerance can mask a real regression that happens to fall just under the threshold.

See Also