Lineage Examples

Getting a first graph out of a Snowflake account nobody documented

Context. A data platform team inherited a Snowflake account with a few hundred tables and views and no data catalog. They know roughly which tables feed the reporting layer, but nothing is written down and the person who built it left. The datastore is registered and synced, so the tables are containers, and every Lineage tab is empty.

What happens. They run a Sync with Collect lineage turned on. Qualytics reads the account's recorded query history, parses the statement text of the queries the history holds, and reads the view definitions. Most reporting tables come back with a full upstream chain in one run.

flowchart LR
    EXT["SALES.PUBLIC.ORDERS_STAGING"]
    RAW["raw_orders"]
    CLEAN["orders_clean"]
    REV["daily_revenue"]

    EXT -.-> RAW
    RAW --> CLEAN
    CLEAN --> REV

The dashed node is the interesting part. ORDERS_STAGING lives in a database nobody registered in Qualytics, so it is drawn as a reference under the name Snowflake spells it with, and the relationship is kept because the other end resolved to a container.

Why it works. Collection answers from what the warehouse actually did, not from what someone remembered to document, so it is the fastest way to a graph that reflects reality. The dashed nodes are a result rather than a defect: they name exactly which assets are part of the pipeline and outside quality coverage. Registering that database and collecting again turns ORDERS_STAGING into an ordinary node, and the chain joins up without the connection being drawn twice.

Keeping bronze, silver, and gold connected when each lives in its own datastore

Context. A team materializes the same logical entity at three stages, and each stage is a separate Qualytics datastore: the raw zone, the curated zone, and the serving zone. Their Atlan instance already knows the whole chain, because the orchestrator publishes it.

What happens. They connect the Atlan integration and run a sync. Qualytics imports the relationships, tagging each connection with the data catalog's name, and the graph for the gold table walks back through silver to bronze across all three datastores.

flowchart LR
    subgraph RAW["Raw datastore"]
        B["customer_bronze"]
    end
    subgraph CUR["Curated datastore"]
        S["customer_silver"]
    end
    subgraph SRV["Serving datastore"]
        G["customer_gold"]
    end

    B --> S
    S --> G

When the orchestrator later drops the silver stage and the data catalog stops publishing that relationship, the next Atlan sync removes it and the graph follows.

Why it works. The data catalog is the source of truth for what it publishes, so reconciling against it is safe: a relationship absent from the data catalog is genuinely absent. That is exactly why the same reconciliation would be wrong for collected lineage, where absence only means the window held no evidence.

Tracing a table in the enrichment datastore back to what it came from

Context. An analyst opens a table in an enrichment datastore whose name they do not recognize, and wants to know which table the rows came from before trusting anything in it.

What happens. The Lineage tab already has a connection, with nothing configured and no sync involved: the scan that wrote the table recorded it. Expanding the fields shows the field-level connections too.

flowchart LR
    subgraph SRC["Source datastore"]
        O["orders"]
        OE["orders.email"]
    end
    subgraph ENR["Enrichment datastore"]
        R["orders_remediation"]
        RE["orders_remediation.email"]
    end

    O -->|written by a scan| R
    OE -->|field level| RE

Why it works. Qualytics knows what it wrote, so this lineage is recorded at the moment of the write rather than derived afterwards. It is also kept rather than reconciled: the rows did flow, and that stays true after a later run stops targeting the same table, which is what lets the analyst trust a connection to a table nobody is writing any more.

Recording a step that happens outside the datastore

Context. A nightly job exports a table to object storage, transforms it with a tool outside the warehouse, and loads the result back. Both tables are in the same Snowflake datastore, and collection connects neither to the other: no statement in the warehouse ever read one to write the other, so there is nothing to observe.

flowchart LR
    E["orders_export"]
    T["Transformation<br/>outside the warehouse"]
    L["orders_enriched"]

    E -.-> T
    T -.-> L
    E ==>|manual connection| L

What happens. A Manager opens the downstream table's Lineage tab and adds the upstream connection by hand. It carries the manual badge and is never touched by a later collection, a data catalog sync, or a computed container update.

Why it works. Every automatic source answers from evidence it can reach, and this hop leaves none in any of them. Manual connections exist for exactly that: knowledge that is real but lives outside every system Qualytics can read. Because nothing else produces it, nothing else removes it either, which is both the point and the cost: if the pipeline changes, only a person will correct it.