Skip to content

Hive Native Troubleshooting

This page documents the known problems with a Hive Native datastore and the steps to resolve them. A Hive Native connection depends on three things in sequence, reaching the metastore, authenticating to it, and reaching the storage the metastore points at, and each section below covers one of them in that order. A failure at the storage step often reads like a broken connection even though the connection itself is fine, so work through the sections top to bottom.

Reaching the Metastore

The Connection Test Reports the Metastore as Unreachable

The Test connection button fails before anything else has been attempted. Qualytics has not authenticated and has not looked at storage yet.

Cause: The Metastore URI is wrong, port 9083 is blocked between your Qualytics deployment and the metastore, or the metastore is not running.

Resolution: Confirm the address, in the form thrift://host:9083, listing every address separated by commas for a highly available metastore. Then allow the Public addresses or Private addresses shown in the Add Datastore form through your security groups or firewall rules, and confirm the metastore service is up.

The Database Dropdown Is Empty or the Sync Finds No Tables

The connection test passes, but the Database dropdown offers nothing, or the first Sync completes with no tables.

Cause: Qualytics reached the metastore, but the identity it presented cannot see the databases. On a Kerberized metastore that identity comes from the keytab on the connection, or from the deployment's credentials when the connection carries none.

Resolution: Grant that identity read access to the Hive databases you want to monitor, then test again. See Permissions.

Authenticating

A failure here means Qualytics reached the metastore and could not establish or present an accepted identity. Check that the realm is reachable, that the keytab is current, and that the Service Principal names the metastore rather than your own account. See Authentication for the distinction between the two identities.

The Connection Is Refused Because a Realm Is Already Registered

Saving or testing the connection fails with a message that its realm is already registered by another datastore.

Cause: Another connection on this deployment uses the same realm name with a different Key Distribution Center. One realm name cannot mean two clusters, so the second connection is refused rather than quietly reading from the wrong one.

Resolution: Confirm which cluster the realm belongs to. Two clusters that share a realm name need distinct names, or separate deployments.

A Kerberos Connection Fails Right After a Credential Change

A connection that worked yesterday fails today, and a credential was changed on the cluster in between.

Cause: The keytab stored on the connection no longer matches the key the Key Distribution Center holds. Qualytics renews tickets from the keytab on its own, so there is nothing to run by hand, but a keytab from before the change cannot renew anything.

Resolution: Upload the current keytab on the connection and test again. The change reaches every datastore that shares the connection.

The Keytab Upload Is Rejected

The Keytab field refuses the file you picked.

Cause: The file is not a keytab, or it is not named with a .keytab extension. Qualytics reads the upload according to its extension, and a keytab is binary. Uploading a krb5.conf into the Keytab field produces the same rejection.

Resolution: Upload the keytab itself, renamed to end in .keytab, and put the krb5.conf in its own field.

An Error Names a Missing Server Principal

Authentication to the metastore succeeds, but reading data fails with an error about a server principal Qualytics could not find.

Cause: Qualytics works out the storage service's principal from the realm of the Service Principal you entered. Your cluster names its storage service differently from that default.

Resolution: Ask your Hadoop administrator for the storage service principal used on the cluster.

Reaching Storage

This is the step that most often looks like something else. Reading table information never touches storage, so a datastore whose storage is unreachable connects cleanly, syncs cleanly, lists every table, and then fails on the first operation that reads records.

The Sync Succeeds but Profiling or Scanning Fails

The connection test and the Sync both pass. The first Profile or Scan fails.

Cause: Table information can be read from the metastore while the data files cannot be reached. Two causes account for almost all of these: the NameNode or DataNode ports are not open to your Qualytics deployment, or the hostnames recorded in the table locations do not resolve outside the cluster. Qualytics follows those locations exactly as the metastore reports them.

Resolution: Allow the NameNode and DataNode ports listed in Permissions, add the storage hostnames to the DNS your deployment uses if they resolve only inside the cluster, and confirm the identity Qualytics presents can read the files, not only list the tables.

A Sync Takes Much Longer Than Expected

The Sync completes, but far more slowly than the size of the database suggests.

Cause: The tables have a very large number of partitions, or storage is responding slowly.

Resolution: Confirm storage is reachable and responsive, then narrow the datastore to the databases you actually need.

Reading Tables

A table Qualytics declines is reported individually, and the rest of the database syncs normally. This is a refusal rather than a failure. Reading an ACID table's files directly would ignore its delete records, and reading a Delta, Hudi, or Iceberg table's files directly would return rows those formats no longer consider current. Refusing is safer than returning data that quietly disagrees with what Hive itself reports.

A Table Is Reported as Unsupported

The Sync completes, but one or more tables appear as unsupported rather than as containers.

Cause: The table is transactional (ACID), a view, backed by a storage handler such as HBase or Kudu, a Delta, Hudi, or Iceberg table registered in Hive, or stored in a format other than Parquet or ORC. The full list is in What Qualytics Reads.

Resolution: Read that table through the Hive connector. Both connectors can point at the same cluster at the same time.

A Scan Is Refused over a Column Name That Differs Only in Case

The Scan stops with a message that a column's name in the data files does not match the one Hive records, apart from its case.

Cause: Hive records column names in lower case while the data files keep the case they were written with. Qualytics confirms the two agree before reading, and refuses the Scan when they differ rather than returning a column full of empty values.

Resolution: Rewrite the affected files with the column names Hive records, or read that table through the Hive connector.

Limitations

  • Parquet and ORC only. Every other format, plus views and transactional tables, is declined rather than read incorrectly.
  • Read-only. Hive Native cannot be used as an enrichment datastore. Link a separate enrichment datastore as its destination for the anomalies and metadata Qualytics produces.
  • Kerberos only. The metastore protocol has no username and password authentication. To connect to Hive as a specific named user, use the Hive connector.
  • One realm name per cluster. Two clusters that use the same realm name with different Key Distribution Centers cannot coexist on one deployment.