Computed File vs Computed Table¶

You reached this page from the Computed Files sub-tree. If you already know your raw data lives in files on object storage, keep reading; if the source is a JDBC warehouse instead, jump to Computed Table vs Computed File.

When a Computed File fits¶

A Computed File is the right choice when:

• The source data lives in files on object storage (Amazon S3, Google Cloud Storage, Azure Data Lake Storage).
• The files are Parquet, ORC, Avro, CSV, TSV, PSV, JSON, Excel, Delta, or Iceberg.
• You want to normalize columns, cast types, or flatten nested arrays (via lateral views + EXPLODE) using Spark SQL.
• You want the transformation cost to scale with your DFS storage layout (partition-aware, columnar-format aware).
• You are OK with a single-source restriction: one source_container_id per Computed File.

When a Computed Table fits instead¶

Switch to a Computed Table if:

• The source data lives in a JDBC warehouse (Snowflake, PostgreSQL, Redshift, SQL Server, Oracle, BigQuery, Databricks, and similar), or in a native datastore (AWS Glue Native, Hive Native, MongoDB, Unity Catalog Native).
• On a JDBC warehouse, you want to reuse the warehouse's own SQL dialect and its query planner, with the query pushed down to the warehouse instead of evaluated by Qualytics's Dataplane. On a native datastore the query is Spark SQL and runs in the Dataplane, as a Computed File does.
• The transformation needs joins across multiple base tables or views in the same datastore.

Side-by-side comparison¶