Lineage Introduction
Overview
Lineage in Qualytics shows how data moves between containers and fields across your datastores. Every container has a dedicated Lineage tab where you can explore those connections visually, add new ones, and trace data quality issues back to their origin.
Why Lineage Matters
A data quality issue rarely stays where it starts. A problem in a raw table can spread to downstream reports and systems that depend on it. Without lineage, finding the root cause and understanding who is affected requires manual investigation across pipelines.
Lineage makes that investigation visual and fast:
- Impact analysis: When an anomaly is detected, lineage shows which downstream consumers may be affected and which upstream sources may be the root cause.
- Pipeline understanding: Team members can navigate a multi-stage pipeline visually instead of reading pipeline code.
- Cross-datastore awareness: When the same logical entity is materialized in multiple datastores at different stages (bronze, silver, gold), lineage keeps those connections visible even when they live in separate datastores.
How Connections Are Created
Connections in Qualytics come from three sources:
- Manual: A user adds a connection directly in the Lineage tab. Once created, it stays until a user removes it explicitly.
- Data catalog: When a data catalog integration syncs, Qualytics imports lineage relationships from the catalog. Each data catalog manages its own connections independently.
- Qualytics managed: Connections generated automatically when a computed container is created or updated. Qualytics detects the sources from the container's definition and keeps connections in sync as the definition changes.
All three types coexist without conflict. A container can have manual, data catalog, and Qualytics managed connections at the same time.
Container-level vs Field-level
Lineage works at two levels of detail:
- Container-level connections link entire containers and show the overall data flow. This is the default view.
- Field-level connections link specific fields between containers, giving you column-level visibility into exactly which fields are involved in each flow and which are affected by a data quality issue.
Both levels coexist in the same graph. You can start with the container view and drill into field-level detail where it matters.
Next Steps
-
How Lineage Works
Container-level vs field-level granularity, why lineage matters for data quality, and how it is gated by the Lineage add-on.
-
Lineage Sources
How each source type is created, updated, and reconciled, including cross-datastore lineage.
-
Reading the Graph
How to navigate the graph, expand connections, and use the toolbar.
-
Field-level Lineage
Expanding field lists, field metadata, and the focal field workflow.