Skip to content

Unity Catalog Native Troubleshooting

This page documents the known problems with a Unity Catalog Native datastore and the steps to resolve them. A Unity Catalog Native connection depends on three things in sequence: reaching the catalog, being accepted by it, and reading the table files the catalog points at. Each section below covers one of them in that order. A failure at the storage step often reads like a broken connection even though the connection itself is fine, so work through the sections top to bottom.

Reaching the Catalog

The Connection Test Reports the Catalog as Unreachable

The Test connection button fails before anything else has been attempted, and the Catalog dropdown stays empty.

Cause: The URL is wrong, or the workspace or Unity Catalog server cannot be reached from your Qualytics deployment.

Resolution: Confirm the address. For Databricks it is the workspace URL, in the form https://<workspace>.cloud.databricks.com, with no path after the host. For an open-source server it is the server's base address. Then allow the Public addresses or Private addresses shown in the Add Datastore form through your security groups or firewall rules, including any IP access list configured on the workspace.

The Catalog or Schema Dropdown Is Empty

The credential is accepted, but the Catalog dropdown offers nothing, or the Schema dropdown offers nothing for the chosen catalog.

Cause: Qualytics reached the catalog, but the identity it presented cannot see any catalog, or cannot see the schemas of that one. The dropdowns show only what the identity holds USE CATALOG and USE SCHEMA on.

Resolution: Grant that identity access to the catalog and the schemas you want to monitor, then test again. You can also type the catalog or schema name instead of picking it, which is useful while the grants are being made. See Permissions.

Authenticating

The Catalog Rejects the Credential

The connection test fails with an authentication error from the catalog.

Cause: The access token is wrong or has expired, or the OAuth client ID and client secret do not match a service principal of that workspace or server.

Resolution: For Access Token, generate a new token in the workspace and save it on the connection. For OAuth (Service Principal), confirm the client ID is the service principal's application ID and that the secret has not expired, then generate a new secret if it has. The change reaches every datastore that shares the connection.

A Connection That Worked Stops Reading on a Given Day

Every datastore on the connection fails at once, on operations that ran fine the day before, with an authentication error.

Cause: The access token on the connection reached its expiry date. Databricks personal access tokens are issued with a lifetime, and nothing renews them.

Resolution: Generate a new token and save it on the connection. To avoid the next expiry, switch the connection to OAuth (Service Principal), which requests short-lived tokens on its own. See Authentication.

The Token Endpoint Cannot Be Reached

With OAuth (Service Principal), the test fails while requesting an access token rather than while talking to the catalog.

Cause: The Token Endpoint points at an address that does not issue tokens for this server, or it is empty and the server's default endpoint, the URL followed by /oidc/v1/token, is not the one this server uses.

Resolution: On Databricks, leave Token Endpoint empty. On an open-source server that delegates sign-in elsewhere, enter the address of the token endpoint it uses.

Reaching Storage

This is the step that most often looks like something else. Reading table definitions never touches storage, so a datastore whose files are unreachable connects cleanly, syncs cleanly, lists every table, and then fails on the first operation that reads records.

The Sync Succeeds but Profiling or Scanning Fails With an Access Error

The connection test and the Sync both pass. The first Profile or Scan fails with an error from the catalog about external access or a missing privilege.

Cause: The identity can read the table definitions but is not allowed to obtain the storage credential for the files. On Databricks, that needs EXTERNAL USE SCHEMA on the schema, and a metastore admin has to have turned on external data access for the metastore. On an open-source server, the server holds no storage credential for the table's location.

Resolution: Grant EXTERNAL USE SCHEMA on each monitored schema and confirm external data access is on for the metastore. On an open-source server, register a storage credential that covers the table locations. See Access in your Unity Catalog.

The Sync Succeeds but Profiling or Scanning Times Out or Is Refused by Storage

The first Profile or Scan fails with a timeout or an access error from the cloud storage itself rather than from the catalog.

Cause: The storage credential was obtained, but the bucket, container, or storage account does not accept traffic from your Qualytics deployment. A credential grants permission, not a network route.

Resolution: Allow the deployment's Public addresses or Private addresses on the storage side, through the bucket policy, the storage firewall, or a private endpoint, and run the operation again. See Network access.

A Long Read of One Table Fails Partway

A Profile or Scan of a very large table runs for about an hour and then fails with an expired credential, while smaller tables in the same operation succeed.

Cause: The storage credential Unity Catalog hands out for a table lasts about an hour, and the read of that one table outlived it. The limit applies per table, since each table gets its own credential.

Resolution: Make the read shorter. Scan the table incrementally so that only new records are read, or add a filter on a partition column. A table partitioned by date and scanned a day at a time stays well inside the window. See Keep reads inside the credential window.

Reading Tables

A Table Is Missing After a Sync or Marked Unloadable

The Sync completes with a warning saying that some tables were skipped, or naming tables that could not be analyzed, and a table you expected is missing from the datastore or carries the Unloadable status.

Cause: The object is a view, a table that Unity Catalog serves only through its Iceberg endpoint, or a foreign table from a federated connection. The native path reads managed and external Delta tables and nothing else. An object the catalog refuses outright is skipped and never becomes a container. A table the catalog lists but Qualytics cannot read is named in the warning and, when it already exists as a container, marked Unloadable. See Supported tables.

Resolution: Read that object through the Databricks connector. Both connectors can point at the same workspace at the same time.

Limitations

  • Delta tables only. Managed and external Delta tables are read. Views, tables reachable only through the Iceberg endpoint, and foreign tables are not. Read them through the Databricks connector.
  • Read-only. Unity Catalog Native cannot be used as an enrichment datastore. Link a separate enrichment datastore as its destination for the anomalies and metadata Qualytics produces.
  • One catalog per connection. The catalog is part of the connection. A second catalog needs a second connection.
  • Reads are bounded by the storage credential. A single read of one table that runs longer than about an hour fails. Scan large tables incrementally or with a partition filter.
  • Storage must be reachable from your deployment. The credential Unity Catalog hands out does not open a network path to the files.