# Data Diff Troubleshooting

Source: https://userguide.qualytics.io/data-quality-checks/data-diff/troubleshooting/

Common problems when creating and running a Data Diff check, their causes, and how to resolve them.

## Validation Fails When Creating the Check

Clicking **Validate** returns an error instead of a success message.

**Cause:** The rule could not run against the selected data. The most common reasons are a field that does not exist on the reference container, an invalid filter clause on either side, or a container that failed to load.

**Resolution:**

1. Review the error message: it explains what needs attention.
2. Confirm every compared field, Row Identifier, and Passthrough Field exists on **both** containers with a comparable type.
3. Check both **Filter Clause** fields for typos: each must be a valid SQL `WHERE` expression for its own container.
4. If the message says a container is marked as **Unloadable**, it was skipped after repeated operation failures. Follow the steps in [Unloadable Container Error](https://userguide.qualytics.io/operations/scan/troubleshooting/#unloadable-container-error).

## Every Row Is Reported as Added and Removed

The anomaly reports the whole dataset twice: once as `added`, once as `removed`.

**Cause:** The two sides are not pairing. Either no **Row Identifiers** are set, or the identifier does not match across the containers: a surrogate key regenerated at load time, a different type on each side, or a value that is formatted differently (padded, trimmed, cased).

**Resolution:** Set Row Identifiers that are stable and identical on both sides. When the identifier is stored differently, normalize it upstream (or with a Computed Field) so the two sides pair on the same value.

## Differences That Do Not Matter Keep Firing

The check reports rows that differ only in trailing decimals, milliseconds, casing, or whitespace.

**Cause:** Without a tolerance, the comparison is exact for every compared field.

**Resolution:** Set the **Comparators** for the affected types: numeric tolerance for amounts and metrics, duration tolerance for timestamps, string normalization for casing and whitespace. Comparators exist exactly for the noise a pipeline legitimately introduces.

## The Anomaly Reports Rows That Only Exist on One Side by Design

A one-way flow reports `added` rows that are expected, because the target keeps receiving new data.

**Cause:** By default all three change types are reported: `added`, `removed`, and `changed`.

**Resolution:** Restrict **Diff Change Types** to the statuses that represent a problem for your flow (often `removed` and `changed`). This requires Row Identifiers, since classification depends on pairing.

## The Two Sides Cover Different Slices

Row counts in the message differ far more than expected, with large `added` and `removed` groups.

**Cause:** The two filters do not scope the same slice. Filtering only the target (or only the reference) makes the extra rows on the unfiltered side look added or removed.

**Resolution:** Keep the two filters aligned, and remember the target filter is applied under **Target** while the reference filter lives on the **Right Reference** panel. The anomaly message does not echo either clause, so open the check itself to confirm the scope each side was read with.

## No Side-by-Side Comparison in the Anomaly

The anomaly opens with the standard Source Records list instead of the paired comparison view.

**Cause:** The Comparison Source Records view requires **Row Identifiers**. Without them the check can only report a set-level difference, with no way to align a target row with a reference row.

**Resolution:** Add Row Identifiers and re-run the scan. See [Comparison Source Records](https://userguide.qualytics.io/anomalies/deep-dive/anomaly-details/source-record/#comparison-source-records) for the full view reference.

## An Edited Check Keeps Behaving the Old Way

You changed the fields, identifiers, or comparators but the anomaly list did not change.

**Cause:** Edits take effect on the next Scan. Saving the check does not re-evaluate the data, and anomalies raised under the previous configuration are not modified.

**Resolution:** Run a Scan on the container (or wait for the scheduled one). Old anomalies stay open until you triage them or a Full scan with [Auto Resolve](https://userguide.qualytics.io/operations/scan/deep-dive/read-strategies/#auto-resolve-on-full-scans) clears them. See [What Happens to Existing Anomalies](https://userguide.qualytics.io/data-quality-checks/data-diff/how-tos/edit-a-check/#what-happens-to-existing-anomalies).

## The Check Is Slow on Large Tables

The scan takes much longer than other checks on the same container.

**Cause:** Data Diff reads both containers and pairs them row by row, so it is the most expensive comparison rule. Wide composite identifiers and long field lists add to that cost.

**Resolution:** Pair on the narrowest stable identifier, compare only the fields that carry business meaning (move the rest to Passthrough Fields), and scope both sides with aligned filters. When per-row detail is not needed, a [Volumetric](https://userguide.qualytics.io/data-quality-checks/volumetric/introduction/) or [Aggregation Comparison](https://userguide.qualytics.io/data-quality-checks/aggregation-comparison/introduction/) check answers "did anything diverge?" at a fraction of the cost.

*Last updated: October 21, 2026.*
