# How After Date Time Checks Work

Source: https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/how-it-works/

## Definition

*Asserts that every value in a Date or Timestamp field is strictly later than a chosen cutoff date and time.*

## Overview

The After Date Time rule defines a lower time boundary on a single date or timestamp field. Each row in the target container must hold a value that is greater than the cutoff; any value equal to or earlier than the cutoff fires an anomaly. The comparison is strict (`>`), not inclusive (`>=`), and the cutoff is stored as a UTC instant.

Typical use cases:

- Enforce a system go-live or migration cutoff.
- Validate ingestion or processing timestamps against a freshness floor.
- Detect stale, replayed, or backfilled rows that slipped through a date-scoped pipeline.

## Field Scope

**Single:** The rule evaluates exactly one field per check.

**Accepted Types**

| Type | Supported |
| --- | --- |
| `Date` | Yes |
| `Timestamp` | Yes |

## General Properties

**Tab: Details**

| Name | Supported |
| --- | --- |
| Filter  Allows the targeting of specific data based on conditions | Supported |
| Coverage Customization  Allows adjusting the percentage of records that must meet the rule's conditions | Supported |

**Tab: Filter Guide**

The filter allows you to define a subset of data upon which the rule will operate.

It requires a valid **Spark SQL** expression that determines the criteria rows in the DataFrame should meet. This means the expression specifies which rows the DataFrame should include based on those criteria. Since it's applied directly to the Spark DataFrame, traditional SQL constructs like WHERE clauses **are not** supported.

> **Example: Examples**
>
> **Direct Conditions**
>
> Simply specify the condition you want to be met.
>
> > **Success: Correct usage" collapsible="true**
> >
> > ```sql
> > O_TOTALPRICE > 1000
> > ```
> >
> > ```sql
> > C_MKTSEGMENT = 'BUILDING'
> > ```
>
> > **Failure: Incorrect usage" collapsible="true**
> >
> > ```sql
> > WHERE O_TOTALPRICE > 1000
> > ```
> >
> > ```sql
> > WHERE C_MKTSEGMENT = 'BUILDING'
> > ```
>
> **Combining Conditions**
>
> Combine multiple conditions using logical operators like `AND` and `OR`.
>
> > **Success: Correct usage" collapsible="true**
> >
> > ```sql
> > O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'
> > ```
> >
> > ```sql
> > (L_SHIPDATE = '1998-09-02' OR L_RECEIPTDATE = '1998-09-01') AND L_RETURNFLAG = 'R'
> > ```
>
> > **Failure: Incorrect usage" collapsible="true**
> >
> > ```sql
> > WHERE O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'
> > ```
> >
> > ```sql
> > O_TOTALPRICE > 1000, O_ORDERSTATUS = 'O'
> > ```
>
> **Utilizing Functions**
>
> Leverage Spark SQL functions to refine and enhance your conditions.
>
> > **Success: Correct usage" collapsible="true**
> >
> > ```sql
> > RIGHT(
> >     O_ORDERPRIORITY,
> >     LENGTH(O_ORDERPRIORITY) - INSTR('-', O_ORDERPRIORITY)
> > ) = 'URGENT'
> > ```
> >
> > ```sql
> > LEVENSHTEIN(C_NAME, 'Supplier#000000001') < 7
> > ```
>
> > **Failure: Incorrect usage" collapsible="true**
> >
> > ```sql
> > RIGHT(
> >     O_ORDERPRIORITY,
> >     LENGTH(O_ORDERPRIORITY) - CHARINDEX('-', O_ORDERPRIORITY)
> > ) = 'URGENT'
> > ```
> >
> > ```sql
> > EDITDISTANCE(C_NAME, 'Supplier#000000001') < 7
> > ```
>
> **Using scan-time variables**
>
> To refer to the current dataframe being analyzed, use the reserved dynamic variable `{{_qualytics_self}}`.
>
> > **Success: Correct usage" collapsible="true**
> >
> > ```sql
> > O_ORDERSTATUS IN (
> >     SELECT DISTINCT O_ORDERSTATUS
> >     FROM {{_qualytics_self}}
> >     WHERE O_TOTALPRICE > 1000
> > )
> > ```
>
> > **Failure: Incorrect usage" collapsible="true**
> >
> > ```sql
> > O_ORDERSTATUS IN (
> >     SELECT DISTINCT O_ORDERSTATUS
> >     FROM ORDERS
> >     WHERE O_TOTALPRICE > 1000
> > )
> > ```
>
> While subqueries can be useful, their application within filters in our context has limitations. For example, directly referencing other containers or the broader target container in such subqueries **is not supported**. Attempting to do so will result in an error.
>
> > **Warning: Important Note on {{_qualytics_self}}**
> >
> > The `{{_qualytics_self}}` keyword refers to the dataframe that's currently under examination. In the context of a full scan, this variable represents the entire target container. However, during incremental scans, it only reflects a subset of the target container, capturing just the incremental data. It's crucial to recognize that in such scenarios, using `{{_qualytics_self}}` may not encompass all entries from the target container.

## Specific Properties

After Date Time has one rule-specific property:

| Name | Description |
| --- | --- |
| Date | The cutoff date and time. The check flags every row whose field value is **not strictly later than** this value. |

## Anomaly Types

| Type | Supported |
| --- | --- |
| Record Flag inconsistencies at the row level | Supported |
| Shape Flag inconsistencies in the overall patterns and distributions of a field | Supported |

## Evaluation Flow

Every After Date Time check follows the same four-step evaluation flow:

1. **Apply the filter clause.** If the check has a `filter` set, only rows matching the filter expression continue to the next step. Rows outside the filter are ignored and cannot contribute to the violation count.
2. **Read the field value as a timestamp.** The platform interprets each row's value as a timestamp for comparison; values that cannot be interpreted are treated as failing the comparison (see below).
3. **Compare against the cutoff.** The cutoff (stored as a UTC instant) is compared to the field value using strict greater-than (`>`). A row passes only when the field value is strictly later than the cutoff.
4. **Apply coverage.** At 100% coverage, any failing row causes the check to fail. Below 100% coverage, the check fails only when the passing fraction drops below the threshold (see [Coverage and Tolerance](https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/how-it-works/#coverage-and-tolerance)).

The comparison is strict, so a row whose field value equals the cutoff is flagged. To get inclusive semantics, set the cutoff earlier than the boundary you want to allow. The form's date and time picker goes down to the minute, so the closest setting there is the preceding minute, which also lets through any value inside that minute. Through the API the cutoff accepts an exact instant, down to fractions of a second, so the window can be made as narrow as the data needs. On a `Date` field, one day earlier is the natural step.

## NULL Handling

The check **passes NULL values**: a row with `NULL` in the evaluated field is not counted as a violation. This is intentional. After Date Time only asserts that *present* values fall after the cutoff and does not enforce mandatory presence on the field.

If the field must also be populated, pair After Date Time with a [Not Null](https://userguide.qualytics.io/data-quality-checks/not-null/introduction/) check on the same field. Together they enforce *"the field must be present AND must be later than the cutoff"*.

Values that cannot be interpreted as a timestamp are **flagged as violations**. Because the field picker accepts only `Date` and `Timestamp` columns, this rarely happens in practice.

## Cutoff and Time Zone Behavior

The cutoff is stored as a UTC instant. When you set the cutoff to `2025-12-01 06:15:00` in the UI, the platform records it as `2025-12-01T06:15:00Z` and uses that exact instant for every evaluation.

The platform evaluates the comparison in UTC:

- `Timestamp` fields stored with explicit time zone offsets are compared directly against the UTC cutoff.
- `Timestamp` fields without an explicit time zone are interpreted as UTC instants before the comparison.
- `Date` fields are interpreted as midnight UTC of that calendar day, then compared to the cutoff instant.

If your source data carries values in mixed or local time zones, normalize the field upstream with a Computed Field before running After Date Time so the comparison is unambiguous.

## The Filter Clause

The filter clause is a SQL `WHERE` expression applied **before** the evaluation. Filtered-out rows are ignored entirely (they cannot trigger a violation and are not counted in the totals).

Common uses:

- Restricting the boundary to a subset of the dataset (`tenant_id = 42`, `status = 'active'`).
- Excluding known-bad legacy rows that are tracked by a separate clean-up task.
- Scoping the check to a partition (`event_date >= '2025-12-01'`).

When a filter is set, the message names it at the end of its first sentence, as `(applied to records where <expression>)`. Record and Shape Anomaly messages both carry it.

## Coverage and Tolerance

Coverage is a fractional value between `0` and `1` that defines the minimum fraction of evaluated rows that must pass:

- **`1.0` (100%, default):** every row in the filtered set must pass. Any failing row causes the check to fail. This is the strictest setting.
- **`< 1.0`:** the check tolerates a fraction of rows failing the comparison. The check fails only when the fraction of passing rows drops below the threshold.

Lower coverage values are useful when a small, known fraction of pre-cutover rows is expected (for example, during a slow migration). Use coverage carefully: a 0.5% tolerance can mask a real regression that happens to fall just under the threshold.

## See Also

- **Anomaly Reporting**

  The anomaly messages the check produces, what the numbers mean, Source Records highlighting, and Custom Anomaly Description.

  [Anomaly Reporting](https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/anomaly-reporting/)
- **Examples**

  Three production scenarios with sample data, anomaly messages, and the SQL equivalent of what the check evaluates.

  [Examples](https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/examples/)
- **Best Practices**

  Guidelines for choosing the cutoff, normalizing time zones, pairing rules, and keeping the signal clean.

  [Best Practices](https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/best-practices/)
- **Permissions**

  The team permission each action needs: view, create, edit, archive, restore, and delete.

  [Permissions](https://userguide.qualytics.io/data-quality-checks/after-date-time/deep-dive/permissions/)

*Last updated: September 22, 2026.*
