Computed Fields FAQ
About Computed Fields
What is a Computed Field?
A Computed Field is a derived field attached to a container. The definition (transformation and source fields it reads) lives with the container's metadata; the value itself is produced when Qualytics profiles or scans the container. The Computed Field appears alongside the container's native fields and can be profiled and monitored the same way.
How is a Computed Field different from a regular field?
Regular fields come from the container's schema: they reflect what the source system actually stores. Computed Fields are declared by you and evaluated by Qualytics on top of that schema. They do not exist in the source system, and deleting a Computed Field never touches the source data.
When should I use a Computed Field instead of a quality check filter?
Use a Computed Field when the derived value needs to be visible on the container, referenced by multiple quality checks, or reported on. Use a check filter when the logic is one-off and only relevant to a single check.
Transformation Choice
Which transformation type should I pick?
Reach for the specialized transformations first: Cast for data-type conversion, Cleaned Entity Name for stripping business signifiers, and Convert Formatted Numeric for formatted-string amounts. Reserve Custom Expression for logic that spans multiple fields, uses conditional branches, or requires window and aggregate functions.
Can I chain Computed Fields?
No. A Computed Field cannot reference another Computed Field. If you need chained logic, combine it into a single Custom Expression.
Does Custom Expression support any Spark SQL function?
Yes. Custom Expression accepts any valid Spark SQL expression against the container's fields, including window functions (LEAD, LAG, ROW_NUMBER), aggregate functions (SUM, AVG, COUNT), and conditional constructs (CASE).
Behavior
When is a Computed Field's value computed?
At Profile and Scan. The transformation runs against the sample the platform reads during Profile, and against the rows Scan pulls to evaluate quality checks. There is no persistent copy of the derived value between runs.
Does saving a Computed Field trigger a profile?
No. Saving persists the definition. The first values populate on the next profile of the container. Trigger a profile manually if you want the field's statistics to appear right away.
What happens when a source field is dropped from the source?
The Computed Field's next profile marks the derived value as Missing. The Computed Field definition itself is preserved, so restoring the source field brings the Computed Field back on the next profile.
What happens when a source field is excluded via Qualytics?
The Computed Field's output is also excluded, but the definition is preserved. Restoring the source field brings the Computed Field back to life. This is different from deleting a Computed Field, which permanently removes both the definition and the output.
Permissions
Who can create a Computed Field?
Any user with the Editor team permission on the parent datastore. An Author on the datastore can also create a Computed Field when owner_id is omitted or set to their own user id (the Hybrid Gate). Reassigning ownership after creation always requires the Editor team permission.
Who can view a Computed Field?
Any user with the Reporter team permission or above on the parent datastore.
Troubleshooting
The Custom Expression fails validation, what should I check?
Confirm every source field referenced in the expression exists on the container by its exact name. Check for typos in function names (Spark SQL is case-sensitive for identifiers when quoted). Malformed CASE, missing END, or unbalanced parentheses are also common causes.
My Computed Field shows Missing after a source update, what should I do?
Open the field's profile page and confirm which source field is missing. If the source field was renamed at the source, edit the Computed Field to reference the new name. If the source field was dropped and needs to come back, restore it at the source and wait for the next Sync and Profile.
Can I recover a deleted Computed Field?
No. Deletion is irreversible. The definition, the derived output, and any quality checks that referenced the field are removed together. If you might need to restore later, exclude the source field instead of deleting the Computed Field.