Computed Fields Introduction

A Computed Field is a derived field attached to a container. Its definition (the transformation and the source fields it reads) lives with the container's metadata; the value itself is produced when Qualytics profiles or scans the container. Once created, the Computed Field appears alongside the container's native fields and can be profiled, referenced by quality checks, and monitored the same way.

Computed Fields let you reshape values without changing the underlying source. A stringified currency amount becomes a real numeric column. Business signifiers like "Inc." or "LLC" are stripped from entity names. A boolean flag is expressed with a Spark SQL CASE statement across three source columns. In every case, the derived field is treated as a regular field on the container.

Why Use One

Common scenarios where a Computed Field is the right tool:

• Type coercion for quality checks. A field ingested as a string needs to be numeric or a timestamp before comparisons and aggregations make sense.
• Standardization. Entity names, product codes, or account labels arrive with inconsistent business signifiers or delimiters. A Computed Field produces a canonical form that quality checks target.
• Derived values. Combine two or more source fields into one (first_name + ' ' + last_name, salary + bonus, hours * rate).
• Categorization. Bucket a continuous value into discrete labels (CASE WHEN revenue > 10000 THEN 'high' ELSE 'low' END).
• Window and aggregate expressions. Express business logic that spans rows within a partition, such as detecting overlapping lease periods with LEAD(...) OVER (...).

Computed Field vs Regular Field

Regular fields come from the container's schema: they are detected when Qualytics profiles the container and reflect what the source system actually stores. Computed Fields are declared by you and evaluated by Qualytics on top of that schema. They do not exist in the source system, and dropping a Computed Field never touches the source data.

Because a Computed Field is derived, its behavior is coupled to its source fields:

• If a source field is renamed at the source and disappears from the profile, the Computed Field's next profile marks it as Missing.
• If a source field is excluded via Qualytics, the Computed Field's output is also excluded but the definition is preserved for restoration.
• Deleting the Computed Field itself removes both the definition and the output permanently.

Transformation Types

Four transformation types cover the common shapes:

• Cast converts a value to a target data type using Spark SQL casting rules.
• Cleaned Entity Name strips business signifiers (prefixes, suffixes, interior tokens) from a string value.
• Convert Formatted Numeric removes non-numeric characters (currency symbols, commas, parentheses) so a formatted string becomes a real numeric value.
• Custom Expression accepts any valid Spark SQL expression across one or more source fields.

See Transformation Types for the full reference.

Next Steps

Deep Dive pages covering the mechanics of Computed Fields: