How the Entity Resolution Recipe Works

The Entity Resolution recipe walks you from a table with suspected duplicates to a golden set in your enrichment destination. Its side panel sums the job up in one sentence: "Find and merge records that describe the same real-world entity, in two stages: set up the resolution check, then assert on your data and remediate the duplicates it finds." This page covers the mechanics that span the whole flow. The steps themselves are detailed on Build and Validate and Scan and Remediate.

Phases and Steps

The recipe runs in two phases. The side panel lists the two phases, and the header above the step content reads Step 1 of 2 or Step 2 of 2 (it counts phases, not steps) followed by the numbered steps of the current phase.

Phase	What it does	Steps
Build & Validate	Configure the entity-resolution check and dry-run it on a sample.	Select Asset, Distinction Field, Compare Fields, Match Threshold, Review, Validate
Scan & Remediate	Full scan, pick golden records, and write the golden set.	Scan, Remediate, Materialize, Complete

Each step opens with a caption that says what it is for, such as "Choose the datastore and table that hold the records to deduplicate." on Select Asset. Back is available on every step after the first, and every step you have already reached stays clickable, so you can revisit any of them. Steps ahead of you lock while the current step's requirement is unmet. Moving forward requires that requirement to be met: an asset selected, a distinction field chosen, at least one compare field, a validated dry run, a completed scan with its records loaded, every entity resolved, and a written golden set.

Changing the configuration after a validation (the asset, the distinction field, the blocking or compare fields, or the threshold) clears the validation and everything downstream of it. The dry run has to be repeated before you can move on to the second phase again.

You can collapse the side panel with Hide steps and bring it back with Show steps. The choice is remembered on your device.

Prerequisites

Two conditions block the flow until they are met.

The table must be profiled. The Select Asset step only lists tables that have been profiled, because the recipe reads the profile to describe fields, compute field metrics, and feed AI suggestions. A datastore without a profiled table shows "No profiled tables in this datastore. Run a profile operation first." (or "files" on a file datastore). Run a Profile operation on the datastore and come back.

The source datastore must have a linked enrichment destination before the Scan step. Both the source records the scan stores for the duplicates it finds and the golden set it later writes need an enrichment destination. When the datastore has none, the Scan step blocks in place with the message "Choose a datastore to receive the scan's records" and a Link Enrichment button. The button opens the same Enrichment Destination dialog as the datastore page, so you can link one without leaving the recipe. See Link an Enrichment Destination from the Scan Step.

Reads and Writes

Your source data is never changed. The recipe reads the asset you pick to identify duplicates, and writes its results elsewhere: the check, scan, and anomaly live in Qualytics, and the golden set and the optional excluded-records output are new containers in your enrichment destination. The How It Works panel on the Select Asset step shows this as a two-column summary: the selected asset under Reads, the golden set and excluded duplicates under Writes, and the reminder "Your table stays exactly as it is".

What the Run Produces

Everything the recipe creates is an ordinary platform object, listed on the Complete step under Everything this run produced and reachable from the pages you already use.

Output	Where it lives	Notes
Entity Resolution check	The checks of the source datastore	Created as a Draft when you leave Review, set to Active when Validate finishes, and left Active afterwards. Its default description is Entity Resolution check on followed by the table's display name.
Dry run	The Validate step	A sampled run over up to 10,000 records per partition. It is not a scan.
Scan operation	The datastore's Activity	A full scan of every record in the selected table, not only the records changed since the last scan.
Shape anomaly	The datastore's Anomalies	Raised by the scan when duplicates exist, one per partition scanned, so a table with a single partition gets one anomaly. Its source records are the duplicates the recipe groups for remediation. Acknowledged automatically when the golden set is written.
Golden set	The enrichment destination	The records you kept from each entity, named `<prefix>_mat_er_<table>`. Each run replaces the output's contents.
Excluded records	The enrichment destination	Optional companion output holding the duplicates you did not keep, named like the golden set with an `_inverse` suffix. Also replaced on each run.

Output names in the enrichment destination

The golden set is named after the enrichment prefix of the source datastore and the source table, in the form <prefix>_mat_er_<table>. With the default prefix _qualytics and a source table called ORDERS, the golden set is _qualytics_mat_er_ORDERS and the excluded records land in _qualytics_mat_er_ORDERS_inverse. When the source table name contains anything other than letters, digits, and underscores, or is longer than 57 characters, the golden set falls back to an id-based name, <prefix>_materialize_er_<id>. The excluded output has less room because of its _inverse suffix and falls back at 49 characters, so for names between 50 and 57 characters the two outputs use different forms. The Materialize step shows both names before you write anything. A JDBC enrichment destination stores them as tables. A DFS enrichment destination lowercases the names and stores them as Delta folders, so the same golden set lands as _qualytics_mat_er_orders.delta.

Inside the enrichment datastore, Qualytics also gives the outputs a readable display name of the form <TABLE> / Entity Resolution (<source datastore>) for the golden set and <TABLE> / Entity Resolution / Excluded (<source datastore>) for the companion, so they are easy to tell apart from plain materialized copies. See Enrichment Introduction for how these outputs reach your destination.

AgentQ's Role

When an AI provider is configured for AgentQ, the recipe offers AI help at defined points. Suggestions appear in a banner at the top of the step, with the reasoning behind each one and an Apply or Apply all action, and you can run them again or hide them. Every step also works without AI: you pick the fields, read the results, and select the golden records yourself.

Where	What AgentQ does	Data sharing level needed
Distinction Field	Suggests the field most likely to identify a record uniquely, weighing uniqueness and completeness across the profiled fields. Runs automatically.	Metadata Shared
Blocking Fields	Suggests safe, exact boundaries that segment possible duplicates. Runs when you open the Blocking Fields view.	Metadata Shared
Compare Fields	Suggests weighted evidence with a comparison type per field, plus a match threshold to go with it. Runs automatically.	Metadata Shared
Validate	Reads the dry-run results and assesses the configuration, suggesting a threshold change when one would help.	Metadata Shared
Scan	Interprets the scan results with a headline, an analysis, and recommended next steps.	Metadata Shared
Remediate	Recommends the best record to keep in each entity. Opt-in: nothing is sent until you click Get AI suggestions, because this step sends the records themselves to the model.	Source Data Shared

The first five steps send statistics and field metadata, never values from your data. The golden-record recommendations are the exception, which is why they sit at the higher level and wait for your consent. When AgentQ's data sharing level is below what a step needs, that step reports that its AI suggestions (or, on Remediate, recommendations) could not load and lets you continue manually; the other AI steps keep working. When no AI provider is configured at all, the AI banners are simply absent. See AgentQ Access Controls for how an administrator sets the level. The global AgentQ chat button is hidden while a recipe is open, since the recipe drives its own suggestions.

Relationship with the Rule Type

The recipe does not change how Entity Resolution works. It only helps you build the check and act on its results. The semantics of blocking and comparison fields, the weighted composite score, the match threshold, NULL handling, and how clusters become entities are documented on How Entity Resolution Checks Work. A check created in the recipe can be edited, cloned, archived, or dry-run like any other, and a check created by hand can be loaded into the recipe. See Entry Points and Resuming.

The How It Works Panel and Working Assets

Two aids travel with you through the recipe.

How It Works is an explainer panel available on the first four steps, opened from the button in the footer. It illustrates the step's concept with a worked example of two customer records: how the distinction field tells records apart, how blocking fields act before any scoring, how compare fields and their weights combine into one weighted score, and how that score is compared with the threshold. On the Match Threshold step the example reacts live to the value you set, flipping between Matched and Not a match.

Working Assets, in the side panel, lists everything the run touches as badges: the datastore and table, the distinction field, the blocking fields, the compare fields, and then the check, the scan, and the anomaly as the recipe creates them. Each badge opens the asset in a new tab, so you can inspect the check or the scan while the recipe stays where you left it. When you enter the recipe from a check or an anomaly, the matching badge flashes to show where you came in.