Lineage Best Practices

Start From What the Source Already Knows

Before adding anything by hand, run a Sync with Collect lineage turned on and see what comes back. Collection reads the query history and the view definitions the source keeps, so it reconstructs the parts of the graph nobody documented, which is usually most of it. Hand-drawn connections are worth the effort only for the hops it cannot see.

Where a data catalog already publishes lineage, connect it too. The two coexist, and neither overwrites the other.

Collect on a Schedule, Not Once

A single collection reads only as far back as the source retains history, and it observes only relationships that were exercised in that window. A table written monthly is invisible to a run that swept a week. Putting Collect lineage on the scheduled Sync closes that gap over time, because each run adds what it sees to what is already there.

Treat Dashed Nodes as a Worklist

A node with a dashed border is an asset the source named and Qualytics does not have. That is a useful signal rather than noise: it names, in the source's own spelling, a table that is part of your pipeline but outside your quality coverage. Register the ones that matter, collect again, and the chain joins up on its own.

Do Not Wait for Collection to Prune

Collected connections are never removed automatically, by design: history can prove that a relationship happened and can never prove that one stopped. A pipeline you decommission leaves its connections behind until someone deletes them. Build that into the decommissioning routine rather than expecting the next run to notice.

Delete at the Source, Not in the Graph

Deleting a connection that something else produces is temporary. A data catalog connection returns on the next sync of that data catalog, a collected one returns the next time the relationship is observed, and a Qualytics managed one returns when the asset is written again. Deleting is the right move for a relationship that is wrong. For one that is merely unwanted, change what produces it.

Reserve Manual Connections for What Nothing Can Observe

A manual connection is never touched by any automatic process, which makes it both durable and unmanaged: it will not be corrected if the pipeline changes. Use it for hops that genuinely leave no trace in any source, such as a transformation that happens outside the datastore, and prefer an automatic source everywhere else.

Go Field-level Where the Transformation Matters

Container-level lineage answers which tables are involved. Field-level lineage answers which column fed which, which is what an impact assessment actually needs when a single column is wrong. Add it for the columns that carry the meaning, rather than trying to mirror every column of every table.

Use the Graph for Impact, Not Just for Documentation

When an anomaly appears, walk upstream for the root cause and downstream for who is affected, before reading any pipeline code. A container with lineage and active anomalies shows both on the same canvas, which is the whole point of keeping the graph current.