Skip to content

How It Works

This page explains the mechanics behind the Export Operation: the asset types it supports, how containers are selected, where the data lands in your enrichment destination, how the operation runs asynchronously, the size limits that apply to each run, and what happens when you re-run it.

Asset types

Each Export Operation captures one asset type per run, chosen at the modal's Choose step:

UI label API value Output name suffix
Anomalies anomalies _export_anomalies
Quality Checks checks _export_checks
Field Profiles profiles _export_field_profiles

The exported columns differ by asset type. See Export Schema for the full column list of each.

Container selection

You decide which containers (tables, files, or views in the source datastore) the export covers:

Selection mode What gets exported
All Every container in the source datastore at the time the operation runs. When attached to a schedule, future containers are included automatically as they are profiled.
Specific Only the containers you select manually. Selection is stable across re-runs.
Tag Every container that carries the selected tag(s) at the time the operation runs. New containers tagged later are picked up by future runs.

The three modes are mutually exclusive: pick one per operation.

Where the data lands

The exported data is written as an output in the linked enrichment destination, not as a browser download. The output name follows a fixed pattern:

_<sanitized-datastore-name>_export_<asset>
  • <sanitized-datastore-name> is the source datastore's name with every non-alphanumeric character replaced by _, then lowercased.
  • <asset> is anomalies, checks, or field_profiles (note: the Field Profiles asset uses the field_profiles suffix in the output name even though the API enum value is profiles).

Examples for a source datastore named "Acme Sales 2024":

  • Anomalies output → _acme_sales_2024_export_anomalies
  • Quality Checks output → _acme_sales_2024_export_checks
  • Field Profiles output → _acme_sales_2024_export_field_profiles

Re-runs overwrite

Running an export a second time for the same datastore and asset type replaces the existing output. The output is not appended to or versioned. If you need point-in-time snapshots, schedule recurring exports and have your downstream pipeline copy the output elsewhere after each run completes.

When Sync and profile the enrichment destination is enabled on the writing datastore, Qualytics discovers and profiles the written containers after the export. When it is disabled, run Sync and then Profile on the destination yourself. See Automatic sync and profile.

Async execution model

The Export Operation runs asynchronously:

  1. You trigger the export. The platform validates the request, creates an Operation record with result = queued, and enqueues the work. The trigger call returns immediately.
  2. The operation moves to result = running. The platform computes the export payload and writes the output to the enrichment destination.

  3. When the write completes, the operation transitions to one of:

    Result Meaning
    success All containers exported.
    partial Some containers exported, others failed. Inspect the operation message for details.
    warning Export completed but with non-fatal issues.
    failure The export did not complete, for example because it exceeded a size limit.

The operation is visible on the source datastore's Activity tab from the moment it is queued. The page auto-refreshes as the result changes.

Export size limits

The platform enforces two limits on every export run while it prepares the data, regardless of the asset type:

Limit Default
Record count 5,000,000 records per export
Payload size 100 MB per export, measured on the prepared export payload (compressed), not the raw output size

An export that exceeds either limit stops and finishes with result = failure. Nothing is written to the enrichment destination, so an output produced by an earlier successful run is left unchanged. The operation message names the limit that was exceeded and asks you to narrow the export scope.

To bring an export under the limits, cover fewer containers per run: switch the container selection from All to Specific or Tag and split the export into several smaller runs. Both limits are set at the deployment level; if a legitimate export keeps exceeding them, ask your administrator (self-hosted) or Qualytics support (managed deployment) about adjusting them.

Scheduling

An Export Operation can be scheduled to run on a recurring cadence. Schedules support:

  • Hourly / Daily / Weekly / Monthly / Custom Cron frequencies
  • Any IANA timezone (e.g. America/New_York, Europe/Paris, Asia/Tokyo). The default is UTC.
  • Automatic Daylight Saving Time handling: a 9:00 AM job in America/New_York runs at 9:00 AM local time year-round.

Deactivating a schedule keeps its cron expression. Reactivating it later resumes the same schedule without re-entering anything.

See Schedule Export for the modal walk-through.

Re-running a completed export

Any user with Editor permission on the source datastore can re-run a previously completed Export Operation. The re-run uses the original operation's parameters (asset type, container selection, include_masked flag) and produces a fresh operation. The new operation overwrites the existing export output on completion.

Re-runs are allowed only after the original operation has finished: attempting to re-run an in-progress operation produces an error.