Entity Resolution Recipe Best Practices

Profile before you start

Only profiled tables are offered on Select Asset, and the field metrics and AI suggestions all read from the profile. Run a fresh profile after a large load so uniqueness and completeness reflect the data you are about to resolve.

Link the enrichment destination first

The Scan step blocks until the source datastore has a linked enrichment destination. Linking it from the datastore page before you begin, with a prefix you recognize, avoids a detour halfway through the flow and gives you a say in where the golden set lands.

Resolve one table at a time

The recipe works on a single asset. When the duplicates span two tables, build a Computed Join from the Select Asset step so the recipe resolves the joined records as one asset.

Pick a truly unique distinction field

The distinction field is the identifier that should tell records apart. Check its Uniqueness and Completeness in the Field Metrics panel: a field with gaps or repeated values makes every entity ambiguous. Surrogate keys and system IDs are the usual choice.

Use blocking fields only for absolute boundaries

A blocking field never lets records with different values be compared, whatever the other fields say. Reserve it for rules you are certain about, such as a tenant or a country when duplicates never cross it. When exceptions are possible, put the field in Compare Fields as an exact comparison instead, where it lowers the score without ruling the pair out.

Combine fuzzy evidence with an exact helper

A fuzzy name alone merges records whose names merely resemble each other. Add at least one independent field, such as an email, a tax ID, or a category as an exact comparison, and give the most reliable field the highest weight in the Weight distribution bar.

Review AI suggestions rather than accepting them blindly

The suggested fields, weights, and threshold are a starting point built from the profile. Read the reasoning attached to each chip, drop fields that do not belong, and adjust weights to what you know about the data.

Start at 0.75 and let the dry run tune it

The default sits in the Balanced band and suits most datasets. Validate, look at the Records Preview, and move the slider from there: loosen it when real duplicates are missing from the sample, tighten it when unrelated records are grouped.

Validate as often as you change something

The dry run runs on a sample and usually takes under a minute, so iterating there is cheap. A full scan is not. Any change to the asset, fields, or threshold invalidates the last validation anyway, so re-validate before moving on.

Read the assessment, then check the preview

Consider adjusting the match threshold and Consider adjusting the compare fields point you at the right step, but the preview shows why. Confirm the diagnosis in the grouped records before applying a suggested threshold.

Let AgentQ propose, then decide

Golden-record recommendations send the records themselves to the model, which is why the step waits for Get AI suggestions. Once you opt in, treat each pick as a proposal: the model chooses on completeness and consistency, while you know which system of record wins.

Keep more than one record when the entity is really several

An entity the check drew too wide is fixed by ticking every record that deserves to survive in the Keep as golden column. That is faster and safer than tightening the threshold for the whole table.

Resolve very large duplicate sets in batches

A single run can exclude at most 25,000 records. When a scan finds more, narrow the check with a filter or resolve part of the data first, materialize, and rerun the recipe on the rest through Scan in Recipe.

Write the excluded records too

The Excluded Records companion output costs one checkbox and gives you an audit trail of every duplicate you dropped, side by side with the golden set. It is the easiest way to answer "what happened to record X" later.

Keep the check active and rerun through the entry points

The check stays active on purpose. After each new load, use Scan in Recipe on the check, or Remediate in Recipe on the anomaly a scheduled scan raised, instead of building the configuration again from the Recipes page.

Fix the source, not just the copy

The golden set is written to the enrichment destination; your source table is never changed. Use the golden set and the excluded records to correct the source system, so the next scan finds fewer duplicates rather than the same ones.

Leave the data sharing level to the administrator

When golden-record recommendations are unavailable, the AgentQ data sharing level is below Source Data Shared. That level is a deliberate decision about sending data values to the AI provider, so raise it with your administrator rather than working around it. Every remediation step works manually in the meantime.