Getting Started with Checks¶
A check applies a rule type to a table or file and turns an expectation about the data into a result Qualytics evaluates on every Scan. The rule type decides what is checked. Everything else on a check, such as the filter, the coverage, the owner, and the tags, works the same way whichever rule you pick.
The Deep Dive pages explain the concepts every check shares, from how a check is evaluated to who creates and owns it. The How-tos walk through each task step by step, for any rule type. The Rule Types section lists every rule in the same groups as the rule type picker on the check form. The API page covers the endpoints, and the FAQ answers the common questions.
Permissions
Anyone with a team permission on a datastore can view its checks. Creating, editing, archiving, and deleting checks need the Drafter or Author team permission, depending on whether the check is a Draft or Active. See Check Permissions for the full matrix.
Deep Dive¶
Understand what a check is, how it is evaluated, who creates and owns it, and what each part of its page shows.
Concepts¶
-
Checks Introduction
Why checks matter and a map of every concept they share.
-
How a Check Works
What a check is made of, how a Scan evaluates it, and how its status changes.
-
Authored Checks
Checks your team writes and governs, which automation never changes.
-
AI Managed Checks
Checks Qualytics generates from observed data, and the edits that make them Authored.
-
Check Ownership
Who is responsible for each check, and how ownership changes.
-
Check Importance
How much a check's failures matter when Qualytics ranks the impact of its anomalies.
-
Impact Metrics
Measure how big a failure is in business terms, such as the amount or the customers affected.
-
Dry Run
Test a check against a sample of its data, without creating anomalies or running a Scan.
Check Details¶
-
Check Views
The check lists, the check page, and the side panel: how to reach each one and what it shows.
-
Check Anomalies
Every anomaly a check has produced, to triage without leaving the check.
-
Check Timeline
Everything that happened to a check: its creation, every change, and your team's comments.
Examples and Permissions¶
-
Check Examples
Scenarios that show ownership and AI Managed checks in practice.
-
Check Permissions
Which team permissions each check action needs.
How-tos¶
Work with checks of any rule type step by step, grouped by what each task changes.
Status¶
-
Activate Draft Checks
Turn a Draft check on, so the next Scan evaluates it.
-
Draft Checks
Pause an Active check without losing its configuration.
-
Archive Checks
Retire a check from Scans and keep it for reference.
-
Restore Archived Checks
Bring an archived check back as Active or Draft.
Managing¶
-
Create a Check
Create a check of any rule type with the check form.
-
Edit Checks
Change one check, or the owner and other settings of many checks at once.
-
Clone a Check
Create a new check from the configuration of an existing one.
-
Delete Checks
Remove archived checks permanently.
-
Mark a Check as Favorite
Keep the checks you follow one click away.
Test and Find Checks¶
-
Dry Run a Check
Test a check against a sample of its data before it goes live.
-
Filter and Sort Checks
Narrow and order any check list.
Rule Types¶
Every rule type, in the same groups as the rule type picker on the check form. Each one has its own section with the full reference, examples, how-tos, and API.
Not sure which rule to pick?
Start from the question you want answered, not from the field. "Is this value inside a range" leads to a different rule than "do these two tables agree", even when both look at the same column.
Single Field¶
Rules that check the values of one field, such as a range, a pattern, a date, or a list of allowed values.
-
After Date Time
Asserts that the field is a timestamp later than a specific date and time.
-
Before Date Time
Asserts that the field is a timestamp earlier than a specific date and time.
-
Between
Asserts that values are equal to or between two numbers.
-
Between Times
Asserts that values are equal to or between two dates or times.
-
Contains Credit Card
Asserts that the values contain a credit card number.
-
Contains Email
Asserts that the values contain email addresses.
-
Contains Social Security Number
Asserts that the values contain social security numbers.
-
Contains Url
Asserts that the values contain valid URLs.
-
Distinct Count
Asserts on the approximate count distinct of the given column.
-
Equal To Field
Asserts that this field is equal to another field.
-
Exists In
Asserts that values assigned to this field exist as values in another field.
-
Expected Values
Asserts that values are contained within a list of expected values.
-
Greater Than
Asserts that the field is a number greater than (or equal to) a value.
-
Greater Than Field
Asserts that this field is greater than another field.
-
Is Address
Asserts that the values are addresses of the required format.
-
Is Credit Card
Asserts that the values are credit card numbers.
-
Is Type
Asserts that the data is of a specific type.
-
Less Than
Asserts that the field is a number less than (or equal to) a value.
-
Less Than Field
Asserts that this field is less than another field.
-
Matches Pattern
Asserts that a field must match a pattern.
-
Max Length
Asserts that a string has a maximum length. On array fields, checks the array has at most N elements. With element context, checks each string element's length.
-
Max Value
Asserts that a field has a maximum value.
-
Min Length
Asserts that a string has a minimum length. On array fields, checks the array has at least N elements. With element context, checks each string element's length.
-
Min Value
Asserts that a field has a minimum value.
-
Not Empty
Asserts that a text or array field is not empty. A value made only of spaces counts as empty.
-
Not Exists In
Asserts that values assigned to this field do not exist as values in another field.
-
Not Future
Asserts that the field's value is not in the future.
-
Not Negative
Asserts that this is a non-negative number.
-
Positive
Asserts that this is a positive number.
-
Predicted By
Asserts that the actual value of a field falls within an expected predicted range.
-
Required Values
Asserts that all of the defined values must be present at least once within a field.
-
Sum
Asserts that the sum of a field is a specific amount.
-
Time Distribution Size
Asserts that the count of records for each interval of a timestamp is between two numbers.
Multiple Fields¶
Rules that take one or more fields together, such as fields that must hold a value, unique keys, or a row-by-row comparison with another table.
-
Any Not Null
Asserts that at least one of the selected fields must hold a value.
-
Data Diff
Asserts that the dataset created by the targeted field(s) has differences compared to the referred field(s).
-
Equal To
Asserts that all of the selected fields equal a value.
-
Is Replica Of
Deprecated. Replaced by Data Diff, which performs the same row-by-row comparison and adds change-type filtering.
-
Not Null
Asserts that the field's values are not null. On array fields with element context, asserts that no elements within the array are null.
-
Unique
Asserts that each field's value is unique across rows. When multiple fields are selected, asserts that the combination of values across those fields is unique (composite key).
Calculated¶
Rules that evaluate an expression you write, compare two aggregations, or match records that describe the same entity.
-
Aggregation Comparison
Verifies that the specified comparison operator evaluates true when applied to the two aggregation expressions.
-
Entity Resolution
Asserts that records describing the same entity are clustered together, even when their values vary, and that every record in a cluster shares one value of a chosen distinction field.
-
Satisfies Expression
Evaluates a SQL expression you provide against every record.
Aggregated¶
Rules that record a value on every Scan and watch how it changes.
-
Metric
Records the value of the selected field during each scan operation and asserts limits based upon an expected change or absolute range (inclusive).
Metadata¶
Rules about the table or file as a whole: how recent it is and how many records it holds.
-
Freshness
Asserts that the data was added or updated within a maximum allowed age.
-
Volumetric
Asserts that the row count of a table or file stays within an expected range, or changes by no more than expected since an earlier day.
Schema¶
Rules about the structure of the table or file rather than the values inside it.
-
Expected Schema
Asserts that all selected fields are present and that all declared data types match expectations.
-
Field Count
Asserts that there must be exactly a specified number of fields.
-
Max Partition Size
Asserts the maximum number of records that should be loaded from each file or table partition.
-
Min Partition Size
Asserts the minimum number of records that should be loaded from each file or table partition.
API & FAQ¶
-
Checks API
Endpoints shared by every rule type: history, versions, insights, ownership, and AI Managed checks.
-
Checks FAQ
Common questions about auditing checks, ownership, and AI Managed checks.