Skip to content

Enrichment FAQ

Answers to common questions about enrichment datastores, linking one as a source datastore's destination, the settings, and what gets written.

Linking

Do I need an enrichment destination to use Qualytics?

No. Sync, Profile, and Scan all run without one. Without a destination, scan results and anomalies exist only inside Qualytics and are not written anywhere you control, and Export and Materialize are unavailable.

Yes, at any time, from the datastore's Settings menu. See Link an Enrichment Destination.

Do I have to create the destination first?

No. The link dialog offers Add new, which creates the datastore in the same dialog, on a new or an existing connection. Creating one needs the Manager role.

Can several datastores share one destination?

Yes. Each keeps its own settings. Use unique enrichment prefixes for Scan, Remediation, and Materialize outputs, and distinct datastore names for Export outputs. The destination's Overview lists every datastore writing to it.

Can I switch to a different destination?

Yes. Open the Enrichment Destination dialog, pick another datastore under Use existing, and save. There is no need to unlink first. Outputs already written stay in the old destination; new writes go to the new one.

Which connectors support enrichment datastores?

Not all of them: a connector has to allow Qualytics to write. See Supported Connectors for the list. A datastore on any connector can still be linked to a destination on another.

Settings

What is the prefix?

The first part of the Scan, Remediation, and Materialize output names Qualytics writes for this datastore. Export output names use the normalized datastore name instead. It is normalized to lowercase with underscores and a leading underscore, so Analytics Bronze becomes _analytics_bronze, and it can be up to 60 characters long.

Does the prefix have to be unique?

Yes, among the datastores writing to the same destination. Qualytics refuses a link or a prefix change that would reuse one.

What happens if I change the prefix?

New outputs are created under the new prefix. The old outputs are neither renamed nor deleted. Remove them directly from the destination if you no longer need them.

What does the remediation strategy control?

Whether a Scan also writes a snapshot of the anomalous records of each container:

  • None (default): no remediation outputs. Source examples are still written.
  • Append: anomalous records are added after every Scan, building a history.
  • Overwrite: the output is replaced with the latest Scan's anomalous records.

What is Maximum Source Examples per Anomaly?

How many rows of source data can be written for each anomaly. New datastores inherit the Platform Default (shipped value 10), from 1 to 1,000,000,000. A Scan can override it. See Source examples.

What is Maximum Record Anomalies per Check?

How many individual anomalies one check may raise per Scan before they are rolled up into a single anomaly. New datastores inherit the Platform Default (shipped value 10), from 1 to 1,000. A Scan can override it.

What does "Sync and profile the enrichment destination" do?

After this datastore writes to the destination, Qualytics runs a Sync of the destination and then a light Profile of the written containers, so they are ready to use without a manual step. It is on by default. Turn it off if you would rather sync the destination yourself. A Sync of the destination that you start by hand, from a schedule, or from a Flow is not followed by that Profile. See Automatic sync and profile.

Can I change the settings after linking?

Yes. Use the Enrichment Destination dialog for the destination, remediation strategy, and automatic Sync/Profile. The prefix is read-only while the current destination is selected; change it when choosing another destination or through the API. Edit the two anomaly limits under Settings > Defaults > Scan. Changes apply to subsequent operations.

What gets written

What happens during a Scan when a destination is linked?

Checks run against the source data; anomalies are detected; failed checks, check metrics, scan metadata, and up to the configured number of source examples are written to the destination; and, under the Append or Overwrite strategy, the anomalous records are written to a remediation output for each container. When Sync and profile the enrichment destination is enabled, Qualytics then discovers and profiles the written containers.

Does enrichment affect Sync or Profile?

Sync and Profile of the source datastore write nothing to the destination. The destination is synced and profiled after supported writes when the source's Sync and profile the enrichment destination setting is on. Export Check Templates is an exception: run Sync and then Profile manually afterward.

Can I run a Scan with a remediation strategy other than None and no destination?

No. Qualytics rejects Append or Overwrite when saving the settings if no destination is linked. Link a destination first. Unlinking or deleting the destination resets the strategy to None.

What do the Scan, Remediation, Materialize, Export badges mean?

They are the container's purpose: which operation wrote it. Scan is what a Scan records, Remediation a snapshot of anomalous records, Materialize a copy of a container, Export exported metadata, Computed a computed asset defined on the destination, and Other a container Qualytics did not write. See Enrichment Purposes.

Why does a container show no last write?

Because Qualytics cannot confirm having written it: the output predates last-write tracking, was written by something else, or its write records were cleaned up. Qualytics shows nothing rather than borrowing the time of the source's most recent operation.

Why is my container called "Failed Checks (TPCH)" instead of its physical name?

That is its display name, built from the purpose and the source so you do not have to decode prefixes. The physical name is shown beside it and is what you query in the destination. See display names.

What happens if I rename a written container?

Qualytics stops maintaining that container's display name and keeps the one you chose, even if the source datastore or source container is later renamed. Clearing the name has the same effect.

How do I see how the outputs join?

Open the destination, click How-tos (or press H), and select Join Enrichment Tables. See View the Relationship Diagram.

The destination

Where do I find the destination in the tree?

In the same tree as every other datastore. The Enrichment filter chip narrows the list to destinations; Source hides them.

Can I build computed assets and run operations on a destination?

Yes. Build computed assets over the written containers, run Sync, Profile, and Scan, and reference the datastore in flows. Checks are also available when you need to validate an individual output.

Why is there no Observability or Quality Score on a destination?

Because the containers there hold Qualytics' own output. Volume and freshness would measure Qualytics' writes rather than your data, and a rollup score would average metadata outputs together with copies of your records. Individual containers can still be profiled and scanned.

What happens to the destination when I delete a source datastore?

The link is removed and the destination is untouched. The outputs the source wrote stay there and keep their purpose badges. Qualytics can no longer name the datastore that wrote them, so they show no source and no last write.

Unlinking

The remediation strategy resets to None, the source's operations stop writing to the destination, and the automatic sync and profile stop for that source. Nothing already written is deleted.

Yes. With the same prefix, the next Scan continues writing to the same outputs.

How do I clean up after unlinking?

Remove the outputs directly from the destination: the four Scan outputs, any remediation outputs, and any materialized or exported outputs written under that prefix. Qualytics never deletes them.

Unlinking needs the Admin role, and it is refused while the source has Export or Materialize operations in a flow or on a schedule. Remove those first.

Storage

How much storage does enrichment use?

It depends on record width, anomaly counts, source-example limits, and how often operations run. The four Scan metadata outputs accumulate across Scans. Append also accumulates remediation records; Overwrite replaces the remediation snapshot but does not limit growth of the Scan metadata outputs. Monitor and manage retention in your storage system.

How do I monitor it?

With your database or cloud provider's own tools. Qualytics does not measure the destination's storage.

Troubleshooting

My Scan failed with an enrichment error. What should I check?

  • No destination linked while the remediation strategy is Append or Overwrite. Link one or set the strategy to None.
  • Write permissions. The destination's connection needs permission to create and update outputs in the target schema or bucket. Each connector page lists the required grants.
  • Destination offline. If the destination cannot be reached, the Scan fails or completes with warnings. Check its connection.

My outputs have a different schema than expected. What happened?

Qualytics manages the schema of the outputs it writes. When a platform version adds a column, the change is applied the next time an operation writes. If the output cannot be updated, that write fails; make sure the destination connection has the required write permissions. See the Schema Changelog.

API

For an existing datastore, linking uses PATCH /api/datastores/{id}/enrichment/{enrichment_id}, which can also set the prefix, and the other settings use PUT /api/datastores/{id}. Bulk creation can supply the destination and settings together. See the Enrichment API and bulk creation.