Computed Tables Introduction¶
A Computed Table is a SQL-defined virtual container that lives inside a JDBC or native source datastore. You describe the shape of the data you want with a single SQL query, and Qualytics stores the query alongside the datastore's other containers. The table is not written back to the warehouse. It is evaluated on demand when Qualytics profiles or scans it.
Once created, a Computed Table behaves exactly like any base table: it appears in the tree view, produces a profile, accepts quality checks, and raises anomalies when its data violates one of those checks.
Why Use One¶
Computed Tables give you a place to prepare data for quality analysis without changing anything in the underlying warehouse. Common scenarios:
- Data preparation and transformation. Clean, shape, or restructure raw data using the SQL dialect of your datastore (Spark SQL on native connectors).
- Complex calculations and aggregations. Compute derived columns, roll-ups, or window functions that would be awkward as standalone quality checks.
- Data subsetting. Narrow a large table down to the rows you actually want to monitor with a
WHEREclause. - Joining tables inside the same datastore. Combine multiple base tables of a single datastore into a single monitored container.
If you need to join tables from different datastores, use a Computed Join instead. A plain Computed Table does not cross datastore boundaries.
What a Computed Table Can Reference¶
A Computed Table can only reference base containers: physical tables and views that already exist in the parent datastore's catalog. It cannot reference:
- Other Computed Tables inside the same datastore.
- Computed Files (those live in DFS datastores: Amazon S3, Google Cloud Storage, or Azure Data Lake Storage).
- Computed Joins.
- Tables outside its parent datastore.
See the How It Works page for the full referencing matrix and the CTE and warehouse-materialization workarounds.
Computed Tables vs Computed Files¶
Computed Tables and Computed Files solve the same problem in different environments: Tables target JDBC warehouses using the warehouse's own SQL dialect, and native datastores using Spark SQL. Files target DFS datastores using Qualytics's own Dataplane. See the Computed Table vs Computed File comparison for a side-by-side.
Next Steps¶
Deep Dive pages covering the mechanics of Computed Tables:
-
How It Works
Execution model, validation semantics, referencing rules, and the fully-qualified-name requirement for certain connectors.
-
Computed Table vs Computed File
Side-by-side comparison: when to reach for a Computed Table over a Computed File.
-
Incremental Scanning
Configure scans to read only rows that changed since the last scan using a timestamp or batch-value field.
-
Cost and Performance
Where the cost of a Computed Table lives and how to keep scans and profiles fast on large data.
-
SQL Dialects per Connector
Connector-specific quirks and the fully-qualified-name list.
-
Permissions
Who can view, create, edit, delete, reassign, and run operations on a Computed Table. Includes the Author-and-owner hybrid gate.
-
Best Practices
A checklist for naming, query style, metadata, profiling, ownership, and sunsetting Computed Tables.
-
Examples
Real-world Computed Tables across e-commerce, finance, SaaS, and data engineering, with the SQL patterns that fit each scenario.
See also: