# Computed Files Introduction

Source: https://userguide.qualytics.io/container/computed-files/deep-dive/introduction/

A **Computed File** is a Spark SQL-defined virtual container that lives inside a DFS source datastore (Amazon S3, Google Cloud Storage, or Azure Data Lake Storage). You describe the shape of the data you want with a Select Expression over a single base file pattern, and Qualytics stores the definition alongside the datastore's other file containers. The file is not written back to storage; it is evaluated on demand when Qualytics profiles or scans it.

Once created, a Computed File behaves exactly like any base file container: it appears in the tree view, produces a profile, accepts quality checks, and raises anomalies when its data violates one of those checks.

## Why Use One

Computed Files give you a place to prepare file data for quality analysis without changing anything in the underlying storage. Common scenarios:

- **Data preparation and transformation.** Clean, shape, or restructure raw files (Parquet, CSV, JSON, Avro, and similar) using Spark SQL.
- **Column-level transformations.** Normalize, cast, split, or combine columns before scanning with Spark SQL functions.
- **Data subsetting.** Narrow a large file pattern down to the rows you actually want to monitor with a `WHERE` clause.
- **Aggregations.** Compute derived columns or roll-ups with `GROUP BY` and standard Spark SQL grouping semantics.

If you need to combine two containers into one, use a [Computed Join](https://userguide.qualytics.io/container/computed-join/getting-started/) instead. A plain Computed File reads from a single source pattern.

## What a Computed File Can Reference

A Computed File can only reference **base file patterns**: physical file patterns already registered under the parent DFS datastore. It cannot reference:

- Other Computed Files inside the same datastore.
- Computed Tables (those live in JDBC and native datastores).
- Computed Joins.
- File patterns outside its parent datastore.
- More than one source pattern at a time.

See the [How It Works](https://userguide.qualytics.io/container/computed-files/deep-dive/how-it-works/) page for the full referencing matrix and the source-file requirements.

## Computed Files vs Computed Tables

Computed Files and [Computed Tables](https://userguide.qualytics.io/container/computed-tables/deep-dive/introduction/) solve the same problem in different environments: Files target DFS datastores using Spark SQL. Tables target JDBC warehouses using the warehouse's own SQL dialect, and native datastores using Spark SQL. See the [Computed File vs Computed Table](https://userguide.qualytics.io/container/computed-files/deep-dive/computed-file-vs-computed-table/) comparison for a side-by-side.

## Next Steps

Deep Dive pages covering the mechanics of Computed Files:

- **How It Works**

  Execution model, validation semantics, and the query editor for Spark SQL over DFS file patterns.

  [How It Works](https://userguide.qualytics.io/container/computed-files/deep-dive/how-it-works/)
- **Computed File vs Computed Table**

  Side-by-side comparison: when to reach for a Computed File over a Computed Table.

  [Computed File vs Computed Table](https://userguide.qualytics.io/container/computed-files/deep-dive/computed-file-vs-computed-table/)
- **Referencing**

  What a Computed File can reference, workarounds for chained transformations, and composing Computed Files into a Computed Join.

  [Referencing](https://userguide.qualytics.io/container/computed-files/deep-dive/referencing/)
- **Lifecycle**

  Profile-on-create, volumetric and freshness tracking, edit re-profiling, history diff, delete cascades, and notifications.

  [Lifecycle](https://userguide.qualytics.io/container/computed-files/deep-dive/lifecycle/)
- **Cost and Performance**

  Where the cost of a Computed File lives and how to keep scans and profiles fast on large file patterns.

  [Cost and Performance](https://userguide.qualytics.io/container/computed-files/deep-dive/cost-and-performance/)
- **Permissions**

  Who can view, create, edit, delete, reassign, and run operations on a Computed File. Includes the Author-and-owner hybrid gate.

  [Permissions](https://userguide.qualytics.io/container/computed-files/deep-dive/permissions/)
- **Best Practices**

  A checklist for naming, query style, metadata, profiling, ownership, and sunsetting Computed Files.

  [Best Practices](https://userguide.qualytics.io/container/computed-files/deep-dive/best-practices/)
- **Examples**

  Real-world Computed Files across JSON log parsing, IoT time-series rollup, array explosion, and event deduplication.

  [Examples](https://userguide.qualytics.io/container/computed-files/deep-dive/examples/)

See also:

- **Add a Computed File**

  Walk through the Add Computed File modal from datastore selection to Save.

  [Add](https://userguide.qualytics.io/container/computed-files/how-tos/add/)
- **API**

  REST endpoints to manage Computed Files programmatically.

  [API](https://userguide.qualytics.io/container/computed-files/api/)
- **FAQ**

  Common questions about references, filters, Spark SQL support, and file-format quirks.

  [FAQ](https://userguide.qualytics.io/container/computed-files/faq/)

*Last updated: November 9, 2026.*