How Anomalies Work¶

Understanding how anomalies are created and change

Anomalies are created by Scans and changed by the people who triage them. This page follows an anomaly from the Scan that finds it to the status that closes it: the checks behind it, what it starts with, how its status can change, how a later Scan recognizes the same issue, and who hears about it along the way.

From Scan to Anomaly¶

DO Running scans that detect anomalies

An anomaly comes at the end of a short chain of operations on a datastore:

1. Connect the datastore. A source datastore and its connection decide which data Qualytics can read.
2. Sync. The Sync operation records the datastore's tables, files, and fields. It does not evaluate any data.
3. Profile. The Profile operation calculates statistics for each field and uses them to create or refresh most AI Managed checks. Volumetric and Freshness checks are maintained from Observability measurements.
4. Add Authored checks. Authored checks cover business rules that observed data alone cannot tell, such as regulatory or contractual limits.
5. Scan. The Scan operation evaluates every active check in its scope. Each failure becomes an anomaly.

The Scan decides which type of anomaly each failure becomes. See Record Anomalies and Shape Anomalies for the details:

• A row that fails one or more record-level checks becomes a Record anomaly.
• A failure that concerns the data as a whole, such as a missing field or a Unique check, becomes a Shape anomaly.
• When a check fails more rows than the Scan's Maximum Record Anomalies per Check (10 by default), all of that check's failures are reported as one Shape anomaly that keeps the total count, and no Record anomaly is created for that check.

When the source datastore has an enrichment datastore, the Scan also writes the rows behind each anomaly there, up to the Scan's Maximum Source Examples per Anomaly. Those are the source records shown on the anomaly. Without an enrichment datastore, the anomaly has no source records. See Scan Settings for both limits.

AI Managed and Authored Checks¶

The checks a Scan evaluates come from two places: AI Managed checks, which Qualytics creates from patterns it observes in the data, and Authored checks, which your team writes for business rules. Both produce anomalies the same way. When every failed check of an anomaly is AI Managed, the anomaly shows an AI badge, and the resolution you archive it with tunes those checks. See AI Managed and Authored Anomalies.

What a New Anomaly Starts With¶

Every new anomaly is filled in from the Scan and the checks that failed:

The tags and assignees are copied once, when the anomaly is created. Changing a check's tags or default assignee later does not change the tags or assignees of the anomalies it already produced. From then on, each anomaly's tags and assignees are edited on the anomaly itself.

Impact is different. While an anomaly is open, its Impact is decided again whenever the Importance of one of its failed checks changes, for example when a tag is added to or removed from the check. Tags on the anomaly itself never change its Impact. See When Impact Updates.

The Lifecycle¶

An anomaly is either open (Active or Acknowledged) or archived (Resolved, Duplicate, Invalid, or Discarded).

flowchart TD
    S([Scan finds a failure]) --> A[Active]
    A -- Acknowledge --> K[Acknowledged]
    K -- Reactivate --> A
    A -- Archive --> R["Archived<br/>Resolved, Duplicate,<br/>Invalid, or Discarded"]
    K -- Archive --> R
    R -- Restore --> K
    R -- Delete --> D([Deleted])

• Open anomalies move freely. An Active anomaly can be acknowledged, and an Acknowledged one can be reactivated. Both stay in the open list.
• Archiving closes the anomaly with a resolution. The resolution says why it was closed. An archived anomaly is read only: its status, tags, assignees, and description cannot change until it is restored.
• Restoring reopens it as Acknowledged. An archived anomaly never goes straight back to Active. To change its resolution, restore it and archive it again.
• Only archived anomalies can be deleted. Deleting removes the anomaly from every list.

People are not the only ones who change a status:

• A Full Scan with Auto Resolve Anomalies turned on resolves open anomalies whose checks ran again and no longer found the same issue. See Auto-Resolve on Full Scans.
• A linked Jira or ServiceNow ticket can move the anomaly when Anomaly status sync is on and the ticket reaches a status mapped to an anomaly status. See anomaly status sync for Jira and ServiceNow.

Every change, by a person, a Scan, or a ticket, is recorded on the anomaly's Timeline. See Anomaly Status for what each status means.

Duplicates and Reactivation¶

DO Running scans with duplicate handling

A Scan that runs again often finds an issue it already reported. Two Scan options, set in the Anomaly Options of the Scan, decide what happens then. Both compare fingerprints:

• Archive Duplicate Anomalies: a new anomaly whose fingerprint matches an open anomaly of the same table or file is archived as Duplicate right away, so the open list keeps only the first one.
• Reactivate Recurring Anomalies: a new anomaly whose fingerprint matches an archived anomaly reopens that anomaly as Active, and the new one is archived as Duplicate. The reopened anomaly keeps its history, tags, and assignees. This option works only together with Archive Duplicate Anomalies.

Two cases are never matched:

• An anomaly archived as Duplicate is never reopened. If the issue comes back, the Scan opens a new anomaly.
• An anomaly without a fingerprint is never marked Duplicate. That is every Freshness and Volumetric anomaly, and every Shape anomaly on a table or file that has no incremental field or last-modified date.

See Scan Settings for both options.

Notifications¶

Anomalies reach people in two ways:

• Flows. A Flow can start when a Scan finds anomalies in a table or file, when it finds an open anomaly, or when an anomaly's status changes, and then send an in-app, email, Slack, Microsoft Teams, or PagerDuty message, or open a ticket. An anomaly archived as Duplicate by the Scan does not start a Flow. See Flow Triggers.
• Assignment notifications. When someone changes the status, tags, description, or assignees of an anomaly, every assignee except that person receives an in-app notification. Creating an anomaly sends none. See Assignment Notification.

A teammate mentioned in a comment on the anomaly also receives a Mention Notification.

See Also¶