Unity Catalog Native FAQ

Answers to common questions about the Unity Catalog Native connector. Sections are grouped by topic. Pick the heading closest to what you are working on.

Choosing a connector

Does Unity Catalog Native replace the Databricks connector?

No. Both stay available, and they are good at different things. Unity Catalog Native reads Delta tables straight from cloud storage with no Databricks compute involved. The Databricks connector runs queries on a cluster or SQL warehouse and can read anything the warehouse can query, including views, and it can also be an enrichment datastore.

Can I connect both against the same workspace?

Yes, at the same time. A common arrangement is Unity Catalog Native for the Delta tables and the Databricks connector for the handful of views alongside them.

Do I have to migrate my existing Databricks datastores?

No. An existing Databricks datastore keeps working exactly as before. Adding Unity Catalog Native is a new datastore on a new connection, not a change to the old one.

Why is there no compute to pay for?

Because nothing runs on Databricks. Qualytics asks the catalog where a table's files are and for permission to read them, then reads the files itself. The only Databricks resource involved is the catalog.

What it reads

Which tables are supported?

Managed and external Delta tables, partitioned or not, in the schema you select. Tables that Unity Catalog makes available only through its Iceberg endpoint are not covered yet, and views are not read because they are queries that only Databricks can run. See Supported tables.

Why is one of my tables marked Unloadable?

Because a Sync could not read a table that was already a container: it was converted to Iceberg, replaced by a foreign table or a view, or its files can no longer be reached. The Sync names it in its warning. Objects the catalog never let Qualytics read, such as views, are not marked Unloadable: the Sync skips them and they never become containers. Read those through the Databricks connector.

Does one unreadable table break the sync?

No. The Sync completes and reports the table in its warning: by name when the catalog listed it but Qualytics could not read it, or as part of a count when the catalog refused it outright. A table that was already a container is marked Unloadable. Every other table in the schema syncs normally.

Can Qualytics write to my catalog?

No. The connector is read-only, so it cannot be used as an enrichment datastore. Link a separate enrichment datastore as its destination for the anomalies and metadata Qualytics produces.

Connecting

Why is the Catalog field in the connection section rather than next to Schema?

Because a connection monitors one catalog. The URL, the credential, and the catalog together say where the connection points, and every datastore on it picks a schema inside that catalog. To monitor a second catalog, create a second connection. The quickest way is Duplicate as a new connection on the existing one, which copies the URL, the authentication type, and the client ID for you to change the catalog. The access token or client secret is not copied, so enter it again, or point the field at a secrets-manager reference that both connections can share.

Can I use it with an open-source Unity Catalog server?

Yes. Enter the server's address as the URL and the token it issued as the credential. The catalog and schema dropdowns fill in from the server, and the server hands out storage credentials for the table files the same way a workspace does, as long as it holds a storage credential for those locations.

The connection test passed and the sync worked, but the profile failed. Why?

Reading table definitions never touches storage, so a datastore whose files are unreachable connects cleanly, syncs cleanly, and then fails on the first operation that reads data. The two usual reasons are an identity that is not allowed to read files from outside Databricks, which needs EXTERNAL USE SCHEMA and external data access on the metastore, and a bucket or storage account that does not accept traffic from your Qualytics deployment. See Troubleshooting.

Why did a very large table fail after about an hour?

The storage credential Unity Catalog hands out for a table lasts about an hour, and a single read of one table that outlives it fails. Scan the table incrementally or with a partition filter so that each read stays inside the window. See Best Practices.

Credentials

Should I use an access token or a service principal?

A service principal with OAuth (Service Principal) where you can. Qualytics then requests short-lived tokens on its own, so there is no expiry date to track, and the service principal belongs to the integration rather than to a person. An access token is quicker to set up and fine for a trial, but every datastore on the connection stops reading the day it expires.

Where do I enter my storage keys?

Nowhere. Qualytics never holds your storage keys. Unity Catalog hands out a short-lived credential for each table's files, which is what makes direct reads safe to allow. See Authentication.

What do I do when the token expires?

Generate a new one and save it on the connection through Manage Connections. Every datastore on the connection picks it up on its next operation.

Can the token come from a secrets manager?

Yes. Turn on Secrets Management on the connection and write a ${key} reference in the Access Token or Client Secret field. See HashiCorp Vault.