Skip to content

Lineage Sources

Every lineage connection in Qualytics records how it was created. There are three categories, each with different rules for how connections are created, updated, and removed:

Category source_type value Created by Updated by Removed by
Manual manual A user in the Lineage tab The same user (or anyone with edit access) A user explicitly deleting the edge
Data catalog import The data catalog name (atlan, alation, collibra, datahub, purview) A data catalog sync The next sync of the same data catalog The next sync of the same data catalog, if the relationship is gone from the data catalog
Qualytics managed inferred A computed container being created or updated A subsequent update to the computed container The computed container being deleted, or its definition no longer referencing the source

Connections from different categories coexist without conflict. A container can have manual, data catalog-imported, and Qualytics managed connections at the same time, and one category never overwrites another. Each data catalog also reconciles independently, so connections imported from Atlan are untouched by an Alation sync and vice versa.

Manual

Manual connections are added directly by a user in the Lineage tab through the Add menu. Both endpoints can be containers or fields.

Manual connections are never modified by background processes. Once you create one, it stays until a user with edit access removes it explicitly.

This is the right choice when:

  • You have lineage knowledge that lives outside any data catalog (for example, a pipeline orchestrated by a custom tool).
  • You want to override or supplement data catalog-imported lineage with connections that the data catalog does not capture.
  • You are documenting field-level relationships that the source systems do not expose.

Data Catalog Import

When a data catalog integration syncs, Qualytics pulls lineage relationships from the data catalog and creates connections tagged with the data catalog's name. The data catalog is the source of truth for these connections, so each sync rebuilds them to match the current state.

Reconciliation is scoped to the data catalog that owns the connection. Each sync:

  • Adds connections that exist in the data catalog but not yet in Qualytics.
  • Removes connections that previously came from the same data catalog but are no longer present.
  • Leaves untouched every connection from a different source. Manual and Qualytics managed connections, and connections imported from other data catalogs, are never affected.

Cross-datastore Lineage

Data catalog import preserves relationships that span multiple datastores. A common pattern is a medallion architecture where the same logical entity is materialized at multiple stages:

  • customer_bronze lives in Datastore A.
  • customer_silver lives in Datastore B.
  • customer_gold lives in Datastore C.

The data catalog knows that the bronze table feeds the silver table, which feeds the gold table, even though each materialization lives in a separate Qualytics datastore. The import keeps those cross-datastore connections intact, so the lineage graph for customer_gold can walk all the way back to customer_bronze regardless of where each container is registered.

Note

Cross-datastore connections only exist when both sides have been registered as containers in Qualytics. If customer_bronze exists in the data catalog but has not been added as a container in any Qualytics datastore, that relationship is skipped.

Supported Data Catalogs

Lineage import is available for every data catalog integration that exposes lineage relationships in its API. For details on configuring each one, see the integration page below.

No Data Catalog
1. Alation Alation
2. Atlan Atlan
3. Collibra Collibra
4. DataHub DataHub
5. Microsoft Purview Microsoft Purview

Qualytics Managed

Qualytics managed connections are generated automatically when a computed container is created or updated. They give you lineage for the assets that Qualytics itself materializes, without requiring a manual entry or a data catalog sync.

Three computed container types produce Qualytics managed connections:

  • Computed table: Sources are detected from the table's SQL definition.
  • Computed join: The left and right containers configured on the join both become sources.
  • Computed file: The source container configured on the computed file becomes the source.

Qualytics managed connections stay in sync with the computed container definition. When you update a computed table and remove a source, the corresponding connection is removed. When you add a new source, a new connection appears.

Note

If a source is not detected automatically, you can still add the connection manually.

Choosing a Source

You do not choose the source directly — it is determined by how the connection is created:

  • Adding a connection from the Lineage tab produces a manual connection.
  • Running a data catalog sync produces connections tagged with that data catalog's name.
  • Saving a computed container produces Qualytics managed connections for the sources in its definition.

If you need a connection that the data catalog does not export and Qualytics does not detect automatically, add it manually.