Insights

Insights summarizes recorded Quality Scores, active checks, Profile and Scan activity, and anomalies for the selected source datastores, report date, and timeframe. Use it to identify changes that need a closer review, then open the related profiles, checks, or anomalies for context.

Navigation

Step 1: Log in to your Qualytics account and click the Explore button on the left side panel of the interface.

You will be navigated to the Insights tab to view a presentation of your data, pulled from the connected source datastore.

Report Date and Timeframe

The Insights tab includes two controls that determine which data is shown on the page:

These controls are specific to the Insights tab. Other Explore tabs (Activity, Profiles, Observability, Checks, Anomalies) do not include them.

Understanding Timeframes and Timeslices

When analyzing data on the Insights, two key concepts help you uncover trends: timeframes and timeslices. These work together to give you both a broad view and a detailed breakdown of your data.

Timeframes

Timeframe is the total range of time you select to view your data. For example, you can choose to see data:

• Weekly: Summarize data for an entire week.

• Monthly: Group data by months.

• Quarterly: Cover three months at a time.

• Yearly: Show data for the entire year.

How Metrics Behave Over a Timeframe

• Quality Score and other similar metrics display an average for the selected timeframe.

Example: If you select weekly, the Quality Score shown will be the average score for the entire week.

• Historical Graphs (like Profiles or Scans) show cumulative totals over time.

Example:If you view a graph for a monthly timeframe, the graph shows how data grows or changes month by month.

Timeslices

Timeslice breaks your selected timeframe into smaller parts. It helps you see more detailed trends within the overall timeframe.

For example:

• A weekly timeframe shows each day of the week.

• A monthly timeframe breaks into weekly segments.

• A quarterly timeframe highlights months within that quarter.

• A yearly timeframe divides into quarters and months.

How Timeslices Work

• When you choose a timeframe, the graph automatically breaks it into timeslices.

• Each bar or point on the graph represents one timeslice.

Example:

• If you choose a Weekly timeframe, each bar in the graph will represent one day of the week.

• If you choose a Monthly timeframe, each bar will represent one week in that month.

Metrics Within a Timeslice

Metrics like Quality Score, Profiles, or Scans are displayed for each timeslice, allowing you to identify trends and patterns over smaller intervals.

Quality Score

Quality Score summarizes recorded quality signals from checks, Profile results, and Observability across Completeness, Coverage, Conformity, Consistency, Precision, Timeliness, Volumetrics, and Accuracy. Use the overall score and dimension percentages as comparable signals within the selected reporting context, then open the underlying checks and anomalies before drawing a conclusion about fitness for use.

Score Summary

This section shows the recorded overall Quality Score and its dimension breakdown. When comparison data is available, it also shows how each score changed from the prior reporting context.

You can hover over any quality dimension (such as Completeness, Precision, or Timeliness) to see additional details about that score.

Score Progression

The Score Progression chart shows how recorded Quality Scores change over time. You can:

• Switch between Day, Week, or Month views.
• Click individual dimensions on the right to see the trend for that specific metric.
• Compare the recorded trend for each dimension.

Use these trends to identify changes that need a closer review.

Data Overview

The Data Overview gives you a high-level snapshot of your data for the selected period. It shows:

• Data Under Management: The total number of rows recorded for management in Qualytics.
• Source Datastores: The number of active source datastores included by the current filters.
• Containers: The number of containers within those datastores.

This helps you quickly understand the scale of your data and how it’s growing over time.

Data Volume

The Data Volume chart shows how recorded volume changes over time for the data included by the current filters. Use it to:

• Spot unexpected growth or drops in data size
• Monitor ingestion patterns
• Compare volume with known ingestion or business cycles
• Identify unexpected changes that need investigation

You can change the Group By option (Day, Week, Month) to view the trend with the level of detail you need.

Checks & Profiling

The Checks & Profiling section provides a consolidated view of your active checks, their status, and profiling activity.

1. Passing Checks: Displays the recorded number of checks that passed in the selected reporting period and filters.

2. Failing Checks: Displays the recorded number of checks that failed in the selected reporting period and filters.

3. Not Asserted Checks: Displays checks without a recorded pass or fail result for the selected reporting context.

4. AI Managed Checks: Displays the recorded number of AI Managed Checks. These checks are generated and maintained by Qualytics AI from observed data behavior and display a purple AI badge on cards and rows.

5. Authored Checks: Displays checks created and governed by your team in the Qualytics platform or API. Authored Checks range from guided rule types and reusable templates to SQL expressions for specialized business logic.

Hover over a category to view its recorded count and ratio for the selected reporting context. Select a category to open the corresponding Checks view in Explore.

6. Records Profiled: This represents the total number of records that were included in the profiling process.

7. Fields Profiled: This shows how many field profiles were updated as a result of the profiling operation.

Anomalies & Scanning

The Anomalies & Scanning section provides a high-level summary of all detected anomalies, along with the scanning activity that helps generate those results. This view helps you understand the number of issues identified and how they are distributed across different statuses, while also showing how much data was scanned during the selected period.

1. Active Anomalies: Shows unresolved anomalies that have not been acknowledged, assigned, or archived. Use severity, ownership, and business impact to prioritize them.

2. Acknowledged Anomalies: Shows anomalies a user has recognized but not yet resolved. Acknowledgement records workflow progress without asserting that the underlying data is correct.

3. Resolved Anomalies: Shows anomalies a user marked Resolved after documenting an outcome. The status records the team's decision; it does not by itself re-evaluate current source data.

4. Duplicate Anomalies: Shows anomalies marked as duplicates of an existing finding so teams can avoid parallel investigation.

5. Invalid Anomalies: Shows anomalies a user determined do not represent a valid issue in their business context.

6. Discarded Anomalies: Shows anomalies intentionally removed from the active workflow because the team decided not to continue investigating them.

Hover over a category to see its recorded count and ratio for the selected report date and time frame. Select a category to open the related anomalies in Explore.

7. Container Scanned: Shows the total number of containers that were scanned during the selected period.

8. Records Scanned: This refers to the number of records that were checked during a scan operation. The scan performs data quality checks on collections like tables, views, and files.

Check Distribution

The Check Distribution section shows a breakdown of checks by rule type, helping you quickly understand which types of rules are most commonly applied across your datastore. Each rule type is represented by a different color for easy visual comparison.

By clicking the caret down 🔽 button, users can choose either the top 5 or top 10 rule types to view in the insights, based on their analysis needs.

Failed Check Distribution

The Failed Check Distribution shows a breakdown of all failed checks by rule type. This helps you quickly understand which rule types are failing most often and may require attention.

By clicking the caret down 🔽 button, users can choose either the top 5 or top 10 rule types to view in the insights, based on their analysis needs.

Profiles

Profiles section provides a clear view of data profiling activities over time, showing how often profiling is performed and the amount of data (records) analyzed.

Profile Runs shows how many times data profiling has been done over a certain period. Each run processes a specific source datastore or table, helping users see how often profiling happens. The graph gives a clear view of the changes in profile runs over time, making it easier to track profiling activity.

Click on the caret down 🔽 button to choose between viewing Records Profiled or Fields Profiled, depending on your preference.

Record Profile

Record Profiled shows the total number of records processed during the profile runs. It provides insight into the amount of data that has been analyzed during those runs. The bars in the graph show the comparison of the number of records profiled over the selected days.

Field Profiled

Field Profiled shows the number of fields processed during the profile runs. It shows how many individual fields within datasets have been analyzed during those runs. The bars in the graph provide a comparison of the fields profiled over the selected days.

Scans

Scans section provides a clear overview of all scanning activities within a selected period. It helps users keep track of how many scans were performed and how many anomalies were detected during those scans. This section makes it easier to understand the scanning process and manage data by offering insight into how often scans occur.

Scan Runs show how often data scans are performed over a certain period. These scans check the quality of data across tables, views, and files, helping users monitor their data regularly and identify any issues. The process can be customized to scan tables or limit the number of records checked, ensuring that data stays accurate and up to standard.

Click on the caret down 🔽 button to choose between viewing Anomalies Identified or Records Scanned, depending on your preference.

Anomalies Identified

Anomalies Identified shows the total number of anomalies detected during the scan runs. The bars in the graph allow users to compare the number of anomalies found across different days, helping them spot trends or irregularities in the data.

Records Scanned

Records Scanned shows the total number of records that were scanned during the scan runs. It gives users insight into how much data has been processed and allows them to compare the scanned records over the selected period.

Export

Export button allows you to quickly download the data from the Insights page. You can export data according to the selected Source Datastores, Tags, Report Date, and Timeframe. This makes it easy to save the data for offline use or share it with others.

After exporting, the data appears in a structured format, making it easy to save for offline use or to share with others.

Refresh

Select Refresh to fetch the latest information currently recorded for the Insights page. Refresh does not run a new Profile or Scan operation.

A label indicates when the page was last refreshed, helping you distinguish the displayed snapshot from newer operation results that may have been recorded since then.