# Entity Matching

Source: https://userguide.qualytics.io/operations/promote/deep-dive/entity-matching/

Every promote operation has to decide, for each source entity, whether a corresponding destination entity already exists or whether a new one must be created. This decision is what **entity matching** does. It is the mechanism that makes re-promotion safe: instead of creating duplicates, the platform finds the destination entity that was produced by a previous promotion and updates it in place.

This page explains the two-phase matching strategy the platform uses, what each phase considers, and how the matching outcome maps to the per-entity result (`created`, `updated`, `skipped`, or `failed`).

## How Promote Decides What to Match

When you promote an entity (quality check, computed field, computed table, or computed file) from a source to a destination, the platform asks: *is there already a destination entity that corresponds to this source?* Two pieces of information drive the answer:

1. **Records of previous promotions.** Every successful promotion (`created` or `updated`) writes a **promote result** row that links the source entity ID to the destination entity ID, scoped to the destination container or datastore. These records are the first thing the platform checks.
2. **Semantic match on the destination.** If no previous promotion is found, the platform falls back to looking at the destination directly, by name for most types, or by attribute overlap for quality checks.

This two-step approach means a re-promotion always finds the destination it produced before, even after edits or renames, while a first-time promotion can still detect and reuse a destination entity that was authored manually.

### Scope Keying Is Asymmetric

The destination scope used to look up a previous promotion differs by promote type:

| Promote Type | Destination Scope | Where the Scope Comes From |
| --- | --- | --- |
| **Quality Checks** | `destination_container_id` | Read from the operation parameters |
| **Computed Fields** | `destination_container_id` | Read from the operation parameters |
| **Computed Tables** | `destination_datastore_id` | Stored as a column on the operation |
| **Computed Files** | `destination_datastore_id` | Stored as a column on the operation |

Container-scoped matching means a quality check promoted to two different containers in the same datastore creates two independent promote result records, one per destination container. Datastore-scoped matching means a computed table promoted to a datastore has a single record regardless of which container name it ends up in.

### Only Successful Results Drive the Lookup

The Phase 1 lookup considers only `created` and `updated` results. `skipped` and `failed` results are not consulted. When the platform finds multiple successful results for the same source entity and destination scope, it uses the **most recent** one (highest result row ID).

## Two-Phase Matching

The platform applies the same two-phase strategy across all promote types.

### Phase 1: Match by Previous Promotion

The platform queries the promote result table for a previous successful promotion of the same source entity to the same destination scope. If a row exists, the recorded destination entity is reused, whether or not names have changed or the source has been edited since.

```mermaid
flowchart TD
    Start([Source Entity]) --> Lookup{Previous promotion<br/>exists?}
    Lookup -- Yes --> Found([Use recorded destination entity])
    Lookup -- No --> Phase2([Proceed to Phase 2])

    style Start fill:#4CAF50,color:#fff,stroke:#388E3C
    style Found fill:#2196F3,color:#fff,stroke:#1565C0
    style Phase2 fill:#FFA726,color:#fff,stroke:#F57C00
```

### Phase 2: Fallback Match

If no previous promotion is found, the platform uses a type-specific strategy to detect a corresponding destination entity that may have been authored manually:

| Promote Type | Fallback Strategy | Match Criteria |
| --- | --- | --- |
| **Quality Checks** | Overlap match | Same rule type, field set, rule-specific properties, and filter (see below) |
| **Computed Fields** | Name match | Same computed field name on the destination container (live rows only) |
| **Computed Tables** | Name match | Same container name on the destination datastore |
| **Computed Files** | Name match | Same container name on the destination datastore |

#### Deleted Destinations

Deleting a computed table or computed file removes it permanently, so a promotion never restores a deleted destination container. If the previously promoted destination was deleted, the next promotion creates a new container with a new ID.

For quality checks and computed fields, the fallback name match runs on **live rows only**. If the previous destination entity has been deleted, the lookup falls through and a new entity is created.

#### Quality Check Overlap Rules

The overlap match for quality checks is precise. Two checks are considered the same if **all** of the following match:

- **Rule type**: exact match.
- **Field set**: the source and destination fields produce the same set when compared case-insensitively.
- **Rule-specific properties**: only the properties that are meaningful for the given rule type are compared. The platform uses an allow-list, which includes (depending on the rule type): `ref_datastore_id`, `ref_container_id`, `window_size`, `ref_expression`, `ref_filter`, `field_name`, `expression`, `interval_name`, `comparison`, `required_labels`, `diff_change_types`, plus an `is_element_context` flag and a coverage indicator (100% vs. less-than-100%).
- **Filter expression**: case-sensitive and whitespace-sensitive equality. A `NULL` filter is treated as the empty string for the purpose of comparison.

Two checks differing only in description, tags, or other metadata are still considered the same under this rule. Overlap matching only ever finds existing destination checks; if no overlap match is found, a new check is created.

> **Info: Matching and updating use different criteria**
>
> The overlap rule above only decides **whether** an existing destination check should be reused. Once that decision is made, the platform runs a separate **definition comparison** to decide between `skipped` and `updated`. That comparison includes additional fields, notably the check **description** and `additional_metadata` (excluding promote-internal keys), so two checks may match under overlap but still trigger an `updated` result if their descriptions or metadata diverge. Tag changes alone never trigger an update.

### Decision After Matching

Once a match is found (through either phase), the platform compares the source and destination definitions:

```mermaid
flowchart TD
    Match{Match found?} -- No --> Create([Create new entity])
    Match -- Yes --> Compare{Definitions identical?}
    Compare -- Yes --> Skip([Skip, no changes needed])
    Compare -- No --> Update([Update destination entity])

    style Create fill:#4CAF50,color:#fff,stroke:#388E3C
    style Skip fill:#9E9E9E,color:#fff,stroke:#616161
    style Update fill:#2196F3,color:#fff,stroke:#1565C0
```

## Why Phase 1 Matters

The Phase 2 fallback alone would be enough for first-time promotions, but it breaks down once the source or destination has been edited. Phase 1 exists so that the source-to-destination identity stays stable across edits, independently of name or matching attributes.

The kinds of edits this protects against differ by entity type:

- **Computed tables, computed files, computed fields.** Renaming the source between promotions. Without a record of a previous promotion, the next promotion would look for a destination with the source's *new* name and create a fresh entity. With one, the platform finds the previously linked destination and updates its name to match.
- **Quality checks.** Quality checks have no user-editable name. A record of a previous promotion instead protects against edits to the **matching attributes**: filter expression, field set, or rule-specific properties. Without one, changing the filter on a previously promoted check would cause the next promotion to overlap-match against nothing and create a duplicate. With one, the destination link is preserved through any source edit.

### Cross-Promotion Consistency

When you promote the same source entity to multiple destinations, each destination scope is matched independently. Re-promoting updates each destination's previously-linked entity without interference.

### Failed Promotion Recovery

If a promotion fails for an entity, the result is recorded as `failed` but does **not** participate in subsequent Phase 1 lookups. On the next promotion attempt, the previous failed result for the same source-destination pair is removed and the new attempt records its result. This means failed promotions can be retried cleanly without stale `failed` rows accumulating against a source entity.

## Promote Metadata

In addition to recording promote results, promoted entities receive metadata on the destination row that records their origin. The exact keys depend on the promote type:

**Tab: Computed Tables and Computed Files**

| Metadata Key | Description |
| --- | --- |
| `promoted_from_datastore` | The source datastore name and ID |
| `promoted_from_container` | The source container name and ID |
| `promote_operation_id` | The ID of the promote operation that created or updated this entity |

**Tab: Computed Fields**

| Metadata Key | Description |
| --- | --- |
| `promoted_from_datastore` | The source datastore name and ID |
| `promoted_from_container` | The source container name and ID |
| `promoted_from_computed_field` | The source computed field name and ID |
| `promote_operation_id` | The ID of the promote operation that created or updated this entity |

**Tab: Quality Checks**

| Metadata Key | Description |
| --- | --- |
| `from_source_datastore` | The source datastore name and ID |
| `from_source_container` | The source container name and ID |
| `from_quality_check_id` | The ID of the source quality check |
| `promote_operation_id` | The ID of the promote operation that created or updated this entity |

This metadata is stored as additional properties on the destination entity and can be queried through the API.

*Last updated: October 16, 2026.*