Skip to content

Computed Table vs Computed File

You reached this page from the Computed Tables sub-tree. If you already know you want a warehouse-backed derived table, keep reading; if the raw data lives in files on object storage, jump to Computed File vs Computed Table instead.

When a Computed Table fits

A Computed Table is the right choice when:

  • The source data lives in a JDBC warehouse (Snowflake, PostgreSQL, Redshift, SQL Server, Oracle, BigQuery, Databricks, and similar), or in a native datastore (AWS Glue Native, Hive Native, MongoDB, Unity Catalog Native).
  • On a JDBC warehouse, you want to reuse the warehouse's own SQL dialect and its query planner (index-aware, statistics-aware, cost-based), with the query running inside the warehouse itself. On a native datastore the query is Spark SQL and runs in Qualytics.
  • The transformation needs joins across multiple base tables or views in the same datastore.
  • You do not need Qualytics to profile the source container before you build the Computed Table.

When a Computed File fits instead

Switch to a Computed File if:

  • The source data lives in files on object storage (Amazon S3, Google Cloud Storage, Azure Data Lake Storage).
  • The files are Parquet, ORC, Avro, CSV, TSV, PSV, JSON, Excel, Delta, or Iceberg.
  • You want to normalize columns, cast types, or flatten nested arrays (via lateral views + EXPLODE) using Spark SQL.
  • You are OK with a single-source restriction (one source_container_id per Computed File, no joins inside the container itself).

Side-by-side comparison

Feature Computed Table (JDBC or native) Computed File (DFS)
Source datastore JDBC warehouse or native datastore DFS (S3, GCS, Azure Data Lake Storage)
Query language Warehouse's own SQL dialect on JDBC, Spark SQL on native Spark SQL
Number of source containers Any base tables and views in the datastore One source_container_id
Joins inside the container Yes, across base tables of the same datastore No, use a Computed Join
Source pre-profile required No Yes, at least once
DISTINCT in the projection Allowed if the warehouse allows it Not allowed, use GROUP BY
Fully-qualified name requirement SQL Server, Oracle, Redshift Not applicable
Execution runtime Warehouse engine on JDBC, Qualytics's Dataplane on native Qualytics's Dataplane
Materialized on storage No (metadata-only definition run on demand) No (metadata-only definition run on demand)