# Data Reconciliation Recipe API

Source: https://userguide.qualytics.io/recipes/data-reconciliation/api/

The Data Reconciliation recipe has no API resource of its own. It is a guided flow over the same objects you can create by hand. Reproducing it programmatically means calling the standard endpoints in the order the recipe does: create the Data Diff check, activate it, scan, materialize the keep and remove sets with your resolutions, and acknowledge the anomaly. The recipe's AI suggestions come from a small set of assist endpoints that you can call too.

> **Tip**
>
> For complete API documentation, including request/response schemas, visit the [API docs](https://userguide.qualytics.io/api/docs).

All endpoints use the base URL of your Qualytics deployment (e.g., `https://your-instance.qualytics.io/api`) and require a Personal API Token. Every call needs the **Member** role. The team permission on the left datastore depends on the step, from **Reporter** for the AI assists to **Editor** for the scan and the materialize, and the right datastore must be readable with **Reporter** or above. See [Permissions](https://userguide.qualytics.io/recipes/data-reconciliation/deep-dive/permissions/) for the full matrix.

> **Note**
>
> The full payload reference for a Data Diff check, including every property, is on the rule type's [API page](https://userguide.qualytics.io/data-quality-checks/data-diff/api/). This page shows only what the recipe sends.

## The Recipe Sequence

### 1. Create the Check as a Draft

The Review step creates the check on the left asset in **Draft** status. The reference asset is named by `ref_datastore_id` and `ref_container_id`, the row identifiers by `id_field_names`, and the compared fields by `fields`. The three comparators are optional and apply per field type; `null` means exact equality.

**Endpoint**: `POST /quality-checks`

**Permission**: **Drafter** team permission on the left datastore for a Draft, **Author** to create it Active directly. The right datastore must be readable with **Reporter** or above.

> **Example: Example request**
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/quality-checks" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "description": "Data Reconciliation check on orders vs orders_source",
>     "container_id": 145,
>     "rule": "dataDiff",
>     "status": "Draft",
>     "fields": ["status", "amount", "currency"],
>     "properties": {
>       "id_field_names": ["order_id"],
>       "passthrough_field_names": ["region"],
>       "ref_datastore_id": 14,
>       "ref_container_id": 301,
>       "ref_filter": null,
>       "diff_change_types": ["added", "removed", "changed"],
>       "numeric_comparator": { "epsilon": 0.01, "as_absolute": true },
>       "duration_comparator": null,
>       "string_comparator": { "ignore_whitespace": true }
>     }
>   }'
> ```
>
> **Response** (200 OK): the created check, with `id` and `status: "Draft"`.

Before creating, the Review step also asks whether an active Data Diff check already covers the same fields on the left asset. The same probe is available as `POST /quality-checks/find-conflicting`, which takes `{"quality_check": <the create payload>}`, needs the **Drafter** team permission, and responds with `conflicting_check` set to the overlapping check or to `null`.

### 2. Save and Activate the Check

The Validate step saves the latest settings to the check and sets it to **Active**. There is no dry run in this recipe. The save replaces `properties` as a whole, so send every property again, as the recipe does. A property left out of the request is cleared.

**Endpoints**: `PUT /quality-checks/{id}` to save, then `PATCH /quality-checks` to activate

**Permission**: **Author** team permission on the left datastore

> **Example: Example requests**
>
> ```bash
> curl -X PUT "https://your-instance.qualytics.io/api/quality-checks/920" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "description": "Data Reconciliation check on orders vs orders_source",
>     "fields": ["status", "amount", "currency"],
>     "properties": {
>       "id_field_names": ["order_id"],
>       "passthrough_field_names": ["region"],
>       "ref_datastore_id": 14,
>       "ref_container_id": 301,
>       "ref_filter": null,
>       "diff_change_types": ["added", "removed", "changed"],
>       "numeric_comparator": { "epsilon": 0.01, "as_absolute": true },
>       "duration_comparator": null,
>       "string_comparator": { "ignore_whitespace": true }
>     }
>   }'
> ```
>
> ```bash
> curl -X PATCH "https://your-instance.qualytics.io/api/quality-checks" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '[{ "id": 920, "status": "Active" }]'
> ```
>
> **Response** (200 OK): the updated check. A save that would overlap with another active check is refused with `409 Conflict` naming that check.

### 3. Run the Full Scan

The Scan step runs a full scan of every record in the left asset, not only the records changed since the last scan, and asserts only the recipe's check through `check_ids`.

**Endpoint**: `POST /operations/run`

**Permission**: **Editor** team permission on the left datastore

> **Example: Example request**
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/operations/run" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "type": "scan",
>     "datastore_id": 12,
>     "container_names": ["orders"],
>     "check_ids": [920],
>     "incremental": false
>   }'
> ```
>
> **Response** (200 OK): the queued scan operation, with its `id`. Poll `GET /operations/{id}` until `result` is no longer `queued` or `running`.

The recipe also raises `enrichment_source_record_limit` so that up to 100,000 mismatched rows are stored for reconciliation, and sets `archive_duplicate_anomalies` to `false`. Once the scan completes, list the check's anomalies with `GET /anomalies?quality_check={check_id}&operation={operation_id}` and read the mismatched rows with `GET /anomalies/{id}/source-record`. Each row carries its change type in the `_qualytics_diff` column and, for changed rows, the fields that differ in `_qualytics_changeset`.

### 4. Materialize the Keep and Remove Sets

The Materialize step runs a materialize operation with a `reconciliation` block: the check, a default side per change type, and your per-row picks as overrides. A `source` verdict keeps the left asset as it is and writes nothing; a `target` verdict writes the row to the keep set (changed and removed rows) or to the remove set (added rows). Rows without a default or an override are treated as `target`.

**Endpoint**: `POST /operations/run`

**Permission**: **Editor** team permission on the left datastore

| Field | Required | Description |
| --- | --- | --- |
| `type` | Yes | `"materialize"`. |
| `datastore_id` | Yes | The left datastore. The outputs are written to its linked enrichment destination. |
| `container_names` | Yes | The left asset, as a one-element list. It must be the check's container. |
| `reconciliation.quality_check_id` | Yes | The Data Diff check. |
| `reconciliation.defaults` | No | The side to apply per change type (`changed`, `added`, `removed`), each `"source"` or `"target"`. |
| `reconciliation.overrides` | No | Per-row picks, each with `id` (the row identifier values) and `verdict`. At most 25,000 overrides, and 1,000,000 characters in total. |

`reconciliation` cannot be combined with `exclusion_field`, `exclusion_values`, `exclusion_filter`, or `materialize_inverse`, which belong to other kinds of materialize.

> **Example: Example request**
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/operations/run" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "type": "materialize",
>     "datastore_id": 12,
>     "container_names": ["orders"],
>     "reconciliation": {
>       "quality_check_id": 920,
>       "defaults": { "changed": "target", "added": "target", "removed": "target" },
>       "overrides": [
>         { "id": { "order_id": "ORD-1041" }, "verdict": "source" }
>       ]
>     }
>   }'
> ```
>
> **Response** (200 OK): the queued materialize operation. When it completes, the enrichment destination holds `<prefix>_mat_dr_orders` and, when rows were listed for deletion, `<prefix>_mat_dr_orders_removals`.

Requests that break the reconciliation rules are rejected with `422 Unprocessable Entity`. The message arrives inside the standard validation error list, prefixed with `Value error,`, or as a plain detail, and reads one of:

- `reconciliation and the exclusion forms are mutually exclusive`
- `materialize_inverse does not apply to a reconciliation materialize`
- `reconciliation.overrides exceeds the maximum count of 25000`
- `reconciliation.overrides exceed the maximum total size of 1000000 characters`
- `reconciliation requires a dataDiff quality check`
- `The reconciliation check does not belong to the requested datastore`
- `A reconciliation materialize must target exactly the check's container`

A materialize on a datastore without a linked enrichment destination, or with a disconnected one, is rejected with `409 Conflict`.

### 5. Acknowledge the Anomaly

After the keep set is written, the recipe acknowledges the anomaly whose mismatches it reconciled. The recipe reads the anomaly's status first and acknowledges it only when it is still Active, or Discarded from an earlier skipped materialize, which it restores. An anomaly archived with any other status stays archived. A direct `PUT` with `Acknowledged` restores any archived anomaly. When nothing was written because every pick kept the left asset, the recipe archives the anomaly as `Discarded` instead.

**Endpoint**: `PUT /anomalies/{id}`

**Permission**: **Author** team permission on the left datastore

> **Example: Example request**
>
> ```bash
> curl -X PUT "https://your-instance.qualytics.io/api/anomalies/5108" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{ "status": "Acknowledged" }'
> ```
>
> **Response** (200 OK): the updated anomaly.

## AI Assist Endpoints

The recipe's AgentQ suggestions come from endpoints under `/recipes/data-reconciliation/`. Each call stands alone: it takes the identifiers of the objects involved, asks the configured AI provider, and returns the suggestion with its reasoning. They all need the **Member** role, the **Reporter** team permission or above on the datastores of the containers, check, or operation named in the request, and an AI integration configured for AgentQ. The AgentQ data sharing level must allow the step: **Metadata Shared** for all of them except resolution recommendations, which need **Source Data Shared**.

| Endpoint | Request | Response |
| --- | --- | --- |
| `POST /recipes/data-reconciliation/suggest-row-identifiers` | `container_id`, `ref_container_id` | `suggested_field_names`, `reasoning` |
| `POST /recipes/data-reconciliation/suggest-compare-columns` | `container_id`, `ref_container_id`, `id_field_names` | `compare_field_names`, `passthrough_field_names`, `numeric_comparator`, `duration_comparator`, `string_comparator`, `reasoning` |
| `POST /recipes/data-reconciliation/analyze-dry-run` | `quality_check_id`, `records_processed`, `added_count`, `removed_count`, `changed_count` | `assessment` (`good`, `adjust_tolerances`, `adjust_identifiers`, or `no_overlap`), `reasoning`, suggested comparators, and the echoed counts. The recipe does not call this endpoint, since it runs no dry run. |
| `POST /recipes/data-reconciliation/interpret-scan-results` | `operation_id`, `quality_check_id`, optional `sampled_added_count`, `sampled_removed_count`, `sampled_changed_count`, `top_changed_fields` | `headline`, `findings`, `recommended_next_steps` |
| `POST /recipes/data-reconciliation/recommend-resolutions` | `quality_check_id`, `mismatches` (each with `key`, `row_status`, `id`, `changed_fields`, `source`, `target`) | `recommendations`, each with `key`, `verdict` (`source` or `target`), `reasoning` |

> **Example: Example: recommend resolutions**
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/recipes/data-reconciliation/recommend-resolutions" \
>   -H "Authorization: Bearer YOUR_QUALYTICS_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "quality_check_id": 920,
>     "mismatches": [
>       {
>         "key": "[\"ORD-1041\"]",
>         "row_status": "changed",
>         "id": { "order_id": "ORD-1041" },
>         "changed_fields": ["amount"],
>         "source": { "order_id": "ORD-1041", "amount": 120.5, "currency": "USD" },
>         "target": { "order_id": "ORD-1041", "amount": 125.0, "currency": "USD" }
>       }
>     ]
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "recommendations": [
>     { "key": "[\"ORD-1041\"]", "verdict": "target", "reasoning": "The reference carries the invoiced amount; the left value looks like a stale pre-discount figure." }
>   ]
> }
> ```

One request may carry at most 25 mismatches; larger sets are sent in several requests, which is how the recipe scores mismatches a few at a time. Mismatches the model could not score are left out of the response rather than failing the request, so a partial response is normal. This endpoint sends both sides' row values to the AI provider; with no AI integration configured it fails and nothing leaves Qualytics.

## Error Responses

| Status Code | Description |
| --- | --- |
| `400 Bad Request` | No AI integration is configured for the assist endpoints, or the requested anomaly status is not allowed, such as moving an archived anomaly back to Active or setting an archived status through this endpoint. |
| `401 Unauthorized` | Missing or invalid API token. |
| `403 Forbidden` | The user lacks the role or the team permission the step needs. |
| `404 Not Found` | No container, check, operation, or anomaly exists with the specified ID. |
| `409 Conflict` | The check would overlap with an active Data Diff check on the same asset, or the left datastore has no linked enrichment destination or a disconnected one, so the materialize cannot run. |
| `422 Unprocessable Entity` | Invalid request body, such as a reconciliation payload that breaks the rules above, a container with no profiled fields, assets that share no comparable column, a row identifier that no shared column qualifies for, or more than 25 mismatches in one request. |
| `429 Too Many Requests` | `Too many concurrent operations. Please try again later` |
| `503 Service Unavailable` | The AI provider could not produce a usable response, or the AgentQ data sharing level does not allow the assist: `AI assist could not produce a usable response. Please try again.` |

> **Example: Error response examples**
>
> **422 Unprocessable Entity** (no comparable columns):
>
> ```json
> { "detail": "The source and target containers share no columns with matching types." }
> ```
>
> **422 Unprocessable Entity** (too many mismatches in one request):
>
> ```json
> { "detail": "Too many mismatches for one request (40); send at most 25." }
> ```
>
> **422 Unprocessable Entity** (materialize on the wrong container):
>
> ```json
> { "detail": "A reconciliation materialize must target exactly the check's container" }
> ```

## Related

- [Data Diff API](https://userguide.qualytics.io/data-quality-checks/data-diff/api/): the full payload and field notes for the check itself.
- [Permissions](https://userguide.qualytics.io/recipes/data-reconciliation/deep-dive/permissions/): the roles and team permissions behind each step.
- [How It Works](https://userguide.qualytics.io/recipes/data-reconciliation/deep-dive/how-it-works/): what the run produces and how the outputs are named.
- [AgentQ Access Controls](https://userguide.qualytics.io/agentq/deep-dive/access-controls/): the data sharing levels that gate the assist endpoints.

*Last updated: October 16, 2026.*
