Observability Introduction

Observability gives you a continuous read on the volume, freshness, and field-level metrics of your data without asking you to write and maintain individual quality checks by hand. It captures measurements automatically or per scan, keeps a rolling history, and surfaces anomalies whenever new data steps outside the expected range.

Why this matters

Data pipelines fail quietly. A source stops producing rows, an ETL lands stale extracts, a numeric field drifts because an upstream calculation changed. Observability catches these silent regressions early by:

• Tracking row counts so unexpected drops or spikes surface as anomalies.
• Tracking freshness so late-arriving data raises a signal instead of going unnoticed.
• Tracking field-level metrics (min, max, average, distinct count, and other measures on a specific field) so you can see how a value's distribution evolves scan by scan.

You can rely on Qualytics to author these checks automatically from accumulated measurements, or you can author them yourself when you want tighter control over the thresholds.

How Observability is organized

The Observability tab sits on three surfaces:

• Datastore scope. Open a Source Datastore and click the Observability tab to see every observability check that belongs to the datastore.
• Container scope. Open a table or file and click the Observability tab to see only that container's checks.
• Explore scope. From the global Explore section, the Observability page lists checks across every datastore you have access to.

At the top of each surface, two tabs group the checks by mechanism:

• Measures. One card per container. Both the Volumetric check (row-count tracking) and the Freshness check (last-modified tracking) live on the same card, so you can see the container's volume and freshness side by side.
• Metrics. One card per metric check. Metric checks target a specific field and compare successive scan results using one of three comparison types.

Read the mechanism deep dives to see how each type is measured, when a measurement counts as an anomaly, and what the chart shows.

Next Steps