Skip to content

Scan Operation

The Scan Operation runs a datastore's data quality checks against its containers (tables, views, or file patterns) and writes every identified anomaly to the linked Enrichment Datastore. Defaults for source examples and record-anomaly limits come from the datastore's Enrichment Settings and can be overridden in the scan form.

Note

The Scan Operation can only run after the Sync and Profile operations have completed for the datastore.

A scan identifies two kinds of anomaly:

  • Record Anomalies: A single record (row) flagged as anomalous, with details on why. The simplest example is a row missing an expected value for a field.

  • Shape Anomalies: Structural issues at the column or schema level, such as missing fields or inconsistent patterns across the dataset.

Within the wizard you can:

  • Choose between an incremental load and a full load.
  • Automatically resolve previously open anomalies that no longer flag on a Full scan.
  • Limit the number of records scanned.
  • Pick which tables or file patterns to include.
  • Schedule the scan to run later.

To open the Scan Operation modal, navigate to a source datastore from the side menu and click the Run button under Scan in the datastore's overview tab. The modal opens at Step 1 (Select Tables) and the stepper at the top shows the full configuration flow.

Scan Operation modal overview

Deep Dive

  • Read Strategies

    Incremental vs Full and Auto-Resolve behavior.

    Read Strategies

  • Scan Settings

    Conceptual reference for every setting in the scan form.

    Scan Settings

  • Permissions

    Who can run, schedule, and configure scans.

    Permissions

How-tos

The numbered cards walk through each step of the wizard. The two unnumbered cards cover post-scan analysis and the API helper for runtime variables.

  • Select Tables

    Choose the containers to scan: All, Specific, or by Tag.

    Select Tables

  • Select Check Categories

    Choose Metadata, Data Integrity, or both.

    Select Check Categories

  • Read Settings

    Pick Incremental or Full, set an optional starting threshold, and the record limit.

    Read Settings

  • Scan Settings

    Anomaly Options (including Auto Resolve), record-anomaly limits, and source examples.

    Scan Settings

  • Schedule Options

    Set up a recurring run, or skip this step and use Run Now.

    Schedule Options

  • Interpret Scan Results

    Walk through the Activity row, the operation detail page, and what the Run looks like in each state.

    Scan Runs by state

  • Use Runtime Variables

    Pass check variables at scan time via the API.

    Use Runtime Variables

Reference

  • Troubleshooting

    Resolution steps for known errors.

    Troubleshooting

  • API

    Payload examples for run, schedule, and retrieve.

    API

  • FAQ

    Common questions.

    FAQ