Manage Anomalies

You can manage anomalies to ensure your data remains accurate and any quality issues are addressed efficiently. Anomalies, which occur due to errors or inconsistencies in data, can be categorized as Open or Archived, helping you track their status and take the appropriate actions. You can acknowledge anomalies that have been reviewed, archive those that no longer need attention, and delete any that are irrelevant or logged by mistake.

Bulk actions further simplify the process, allowing you to manage multiple anomalies at once, saving time and effort. This guide will walk you through the steps of acknowledging, archiving, restoring, editing, and deleting anomalies, whether individually or in bulk.

Let's get started 🚀

Navigation

Step 1: Log in to your Qualytics account and select the datastore from the left menu on which you want to manage your anomalies.

Step 2: Click on the “Anomalies” from the Navigation Tab.

Categories Anomalies

Anomalies can be classified into different categories based on their status and actions taken. These categories include Open and Archived anomalies. Managing anomalies effectively helps in maintaining data integrity and ensuring quick response to issues.

Open

By selecting Open Anomalies, you can view anomalies that have been detected but remain unacknowledged or unresolved. These anomalies require attention and may need further investigation or corrective action.

This option helps focus on unaddressed issues while allowing seamless navigation to All, Active, or Acknowledged anomalies as needed.

1. Active: By selecting Active Anomalies, you can focus on anomalies that are currently unresolved or require immediate attention. These are the anomalies that are still in play and have not yet been acknowledged, archived, or resolved.

2. Acknowledged: By selecting Acknowledged Anomalies, you can see all anomalies that have been reviewed and marked as acknowledged. This status indicates that the anomalies have been noted, though they may still require further action.

3. All: By selecting All Anomalies, you can view the complete list of anomalies, regardless of their status. This option helps you get a comprehensive overview of all issues that have been detected, whether they are currently active, acknowledged, or archived.

Archived

By selecting Archived Anomalies, you can view anomalies that have been resolved or moved out of active consideration. Archiving anomalies allows you to keep a record of past issues without cluttering the active list.

You can also categorize the archived anomalies based on their status as Resolved, Duplicate and Invalid, to manage and review them effectively.

1. Resolved: This indicates that the anomaly was a legitimate data quality concern and has been addressed.

2. Duplicate: This indicates that the anomaly is a duplicate of an existing record and has already been addressed.

3. Invalid: This indicates that the anomaly is not a legitimate data quality concern and does not require further action.

4. All: Displays all archived anomalies, including those marked as Resolved, Duplicate, and Invalid, giving a comprehensive view of all past issues.

Acknowledge Anomalies

By acknowledging anomalies, you indicate that they have been reviewed or recognized. This can be done either individually or in bulk, depending on your workflow. Acknowledging anomalies helps you keep track of issues that have been addressed, even if further action is still required.

Warning

Once an anomaly is acknowledged, it remains acknowledged and never reverts to the active state.

Method I. Acknowledge Specific Anomaly

You can acknowledge individual anomalies either directly or through the action menu, giving you precise control over each anomaly's status.

1. Acknowledge Directly

Step 1: Locate the active anomaly you want to acknowledge, and click on the acknowledge icon (represented by an eye icon) located on the right side of the anomaly.

Step 2: After clicking on the Acknowledge button your anomaly is successfully moved to the acknowledge and a flash message will appear saying “The Anomaly has been successfully acknowledged.”

2. Acknowledge via Action Menu

Step 1: Click on the active anomaly from the list of available anomalies that you want to acknowledge.

Step 2: A modal window will appear displaying the anomaly details. Click on the Acknowledge icon ( represented by an eye icon ) located in the upper-right corner of the modal window.

Step 3: After clicking on the Acknowledge button your anomaly is successfully moved to the acknowledge and a flash message will appear saying “The Anomaly has been successfully acknowledged.”

Method II. Acknowledge Anomalies in Bulk

By acknowledging anomalies in bulk, you can quickly mark multiple anomalies as reviewed at once, saving time and ensuring that all relevant issues are addressed simultaneously.

Step 1: Hover over the active anomalies and click on the checkbox to select multiple anomalies.

When multiple anomalies are selected, an action toolbar appears, displaying the total number of selected anomalies along with a vertical ellipsis for additional bulk action options.

Step 2: Click on the vertical ellipsis (⋮) and choose "Acknowledge" from the dropdown menu to acknowledge the selected anomalies.

A modal window titled “Acknowledge Anomalies” will appear, confirming that this action acknowledges the anomalies as a legitimate data quality concern.

You also have the option to leave a comment in the provided field to provide additional context or details.

Step 3: Click on the Acknowledge button to acknowledge the anomalies.

Step 4: After clicking on the Acknowledge button your anomalies are successfully moved to the acknowledge state and a flash message will appear saying “The Anomalies have been successfully acknowledged”

Archive Anomalies

By archiving anomalies, you move them to an inactive state, while still keeping them available for future reference or analysis. Archiving helps keep your active anomaly list clean without permanently deleting the records.

Method I. Archive Specific Anomalies

You can archive individual anomalies either directly or through the action menu.

1. Archive Directly

Step 1: Locate the anomaly (whether Active or Acknowledged) you want to archive and click on the Archive icon (represented by a box with a downward arrow) located on the right side of the anomaly.

Step 2: A modal window titled “Archive Anomaly” will appear, providing you with the following archive options:

• Resolved: Choose this option if the anomaly was a legitimate data quality concern and has been addressed. This helps maintain a record of resolved issues while ensuring that they are no longer active.

• Duplicate: Choose this option if the anomaly is a duplicate of an existing record and has already been addressed. No further action is required as the issue has been previously resolved.

• Invalid: Select this option if the anomaly is not a legitimate data quality concern and does not require further action. Archiving anomalies as invalid helps differentiate between real issues and those that can be dismissed, improving overall data quality management.

Step 3: Once you've made your selection, click the Archive button to proceed.

Step 4: After clicking on the Archive button your anomaly is moved to the archive and a flash message will appear saying “Anomaly has been successfully archived”

2. Archive via Action Menu

Step 1: Click on the anomaly from the list of available (whether Active or Acknowledged) anomalies that you want to archive.

Step 2: A modal window will appear displaying the anomaly details. Click on the Archive icon ( represented by a box with a downward arrow ) located in the upper-right corner of the modal window.

Step 3: A modal window titled “Archive Anomaly” will appear, providing you with the following archive options:

• Resolved: Choose this option if the anomaly was a legitimate data quality concern and has been addressed. This helps maintain a record of resolved issues while ensuring that they are no longer active.

• Duplicate: Choose this option if the anomaly is a duplicate of an existing record and has already been addressed. No further action is required as the issue has been previously resolved.

• Invalid: Select this option if the anomaly is not a legitimate data quality concern and does not require further action. Archiving anomalies as invalid helps differentiate between real issues and those that can be dismissed, improving overall data quality management.

Step 4: Once you've made your selection, click the Archive button to proceed.

Step 5: After clicking on the Archive button your anomaly is moved to the archive and a flash message will appear saying “Anomaly has been successfully archived”

Method II. Archive Anomalies in Bulk

To handle multiple anomalies efficiently, you can archive them in bulk, allowing you to quickly move large volumes of anomalies into the archived state.

Step 1: Hover over the anomaly (whether Active or Acknowledged) and click on the checkbox to select multiple anomalies.

When multiple anomalies are selected, an action toolbar appears, displaying the total number of selected anomalies along with a vertical ellipsis for additional bulk action options.

Step 2: Click on the vertical ellipsis (⋮) and choose "Archive" from the dropdown menu to archive the selected anomalies.

Step 3: A modal window titled “Archive Anomaly” will appear, providing you with the following archive options:

• Resolved: Choose this option if the anomaly was a legitimate data quality concern and has been addressed. This helps maintain a record of resolved issues while ensuring that they are no longer active.

• Duplicate: Choose this option if the anomaly is a duplicate of an existing record and has already been addressed. No further action is required as the issue has been previously resolved.

• Invalid: Select this option if the anomaly is not a legitimate data quality concern and does not require further action. Archiving anomalies as invalid helps differentiate between real issues and those that can be dismissed, improving overall data quality management.

Step 4: Once you've made your selection, click on the Archive button to proceed.

Step 5: After clicking on the Archive button your anomaly is moved to the archive and a flash message will appear saying “Anomalies has been successfully archived”

Restore Archive Anomalies

By restoring archived anomalies, you can bring them back into the acknowledged state for further investigation or review. These anomalies will not return to the active state once they have been acknowledged.

Step 1: Click on the anomaly that you want to restore from the list of archived anomalies.

Step 2: A modal window will appear displaying the anomaly details. Click on the vertical ellipsis (⋮) located in the upper-right corner of the modal window, and click on “Restore” from the drop-down menu.

Step 3: After clicking on the “Restore” button, the selected anomaly is now restored as in acknowledged state.

Edit Anomalies

By editing anomalies, you can only update their tags, allowing you to categorize and organize anomalies more effectively without altering their core details.

Note

When editing multiple anomalies in bulk, only the tags can be modified.

Step 1: Hover over the anomaly (whether Active or Acknowledged) and click on the checkbox.

You can edit multiple checks by selecting the checkboxes next to each anomaly to choose multiple anomalies at once.

When multiple anomalies are selected, an action toolbar appears, displaying the total number of selected anomalies along with a vertical ellipsis for additional bulk action options.

Step 2: Click on the vertical ellipsis (⋮) and choose "Edit" from the dropdown menu to edit the selected anomalies.

A modal window titled “Bulk Edit Anomalies” will appear. Here you can only modify the “tags” of the selected anomalies.

Step 3: Turn on the toggle and assign tags to the selected anomalies.

Step 4: Once you have assigned the tags, click on the “Save” button.

After clicking the Save button, the selected anomalies will be updated with the assigned tags.

Delete Anomalies

Deleting anomalies allows you to permanently remove records that are no longer relevant or were logged in error. This can be done individually or for multiple anomalies at once, ensuring that your anomaly records remain clean and up to date.

Note

You can only delete archived anomalies, not active or acknowledged checks. If you want to delete an active or acknowledged anomaly, you must first move it to the archive, and then you can delete it.

Warning

Deleting an anomaly is a one-time action. It cannot be restored after deletion.

Method I. Delete Specific Anomaly

You can delete individual anomalies using one of two methods:

1. Delete Directly

Step 1: Click on Archived from the navigation bar in the Anomalies section to view all archived anomalies.

Step 2: Locate the anomaly, that you want to delete and click on the Delete icon located on the right side of the anomaly.

Step 3: A confirmation modal window will appear, click on the Delete button to permanently remove the anomaly from the system.

Step 4: After clicking on the delete button, your anomaly is successfully deleted and a success flash message will appear saying “Anomaly has been successfully deleted”

2. Delete via Action Menu

Step 1: Click on the archive anomaly from the list of archived anomalies that you want to delete.

Step 2: A modal window will appear displaying the anomaly details. Click on the vertical ellipsis (⋮) located in the upper-right corner of the modal window, and click on “Delete” from the drop-down menu.

Step 3: A confirmation modal window will appear, click on the Delete button to permanently remove the anomaly from the system.

Step 4: After clicking on the delete button, your anomaly is successfully deleted and a success flash message will appear saying “Anomaly has been successfully deleted”

Method II. Delete Anomalies in Bulk

For more efficient management, you can delete multiple anomalies at once using the bulk delete option, allowing for faster cleanup of unwanted records.

Step 1: Hover over the archived anomalies and click on the checkbox to select anomalies in bulk.

When multiple checks are selected, an action toolbar appears, displaying the total number of checks chosen along with a vertical ellipsis for additional bulk action options.

Step 2: Click on the vertical ellipsis (⋮) and choose "Delete" from the dropdown menu to delete the selected anomalies.

Step 3: A confirmation modal window will appear, click on the Delete button to permanently remove the selected anomalies from the system.

Step 4: After clicking on the delete button, your anomalies are successfully deleted and a success flash message will appear saying “Anomalies has been successfully deleted”

Filter and Sort

Filter and Sort options allow you to organize your anomaly by various criteria, such as Weight, Anomalous Record, Created Date. You can also apply filters to refine your list of anomaly based on Timeframe, Type and Rule etc.

Sort

You can sort your anomalies by Anomalous Record, Created Date, and Weight to easily organize and prioritize them according to your needs.

No	Sort By Option	Description
1	Anomalous Record	Sorts anomalies based on the number of anomalous records identified.
2	Created Date	Sorts anomalies according to the date they were detected.
3	Weight	Sort anomalies by their assigned weight or importance level.

Whatever sorting option is selected, you can arrange the data either in ascending or descending order by clicking the caret button next to the selected sorting criteria.

Filter

You can filter your anomalies based on values like Timeframe, Type, Rule, and Tags etc.

No.	Filter	Description
1	Timeframe	Filtering anomalies detected within specific time ranges (e.g., anomalies detected in the last week or year).
2	Type	Filter anomalies based on anomaly type (Record or Shape).
3	Rule	Filter anomalies based on specific rules applied to the anomaly. By clicking on the caret down button next to the Rule field, the available rule types will be dynamically populated based on the rule types present in the results. The rules displayed are based on the current dataset and provide more granular control over filtering. Each rule type will show a counter next to it, displaying the total number of occurrences for that rule in the dataset. For example, the rule type Between is displayed with a total of 13 occurrences.
4	Table	Filters anomalies based on the table where they occurred.
5	Field	Filters anomalies based on the column in the table where the issue was found.
6	Check	Filters anomalies based on the check that generated them.
7	Tags	Filters anomalies by tags that have been applied to checks or rules.