Skip to content

Hive Native FAQ

Answers to common questions about the Hive Native connector. Sections are grouped by topic; pick the heading closest to what you are working on.

Choosing a connector

Does Hive Native replace the Hive connector?

No. Both stay available, and they are good at different things. Hive Native is the faster path for file-based tables, including Parquet, ORC, delimited text, JSON, and Avro; the Hive connector reads anything Hive itself can query, including views and transactional tables.

Can I connect both against the same cluster?

Yes, at the same time. A common arrangement is Hive Native for the file-based tables and the Hive connector for the handful of views or transactional tables alongside them.

Do I have to migrate my existing Hive datastores?

No. An existing Hive datastore keeps working exactly as before. Adding Hive Native is a new datastore on a new connection, not a change to the old one.

Why is it faster?

It leaves the cluster's query engine out of the read path. Instead of sending a query to HiveServer2 and waiting for the rows to come back, Qualytics reads the table's files directly. On a large table that difference is considerable.


What it reads

Which table formats are supported?

Parquet, ORC, delimited text, JSON, and Avro, plus other formats Hive reads through a SerDe available to Qualytics, such as RegexSerDe, SequenceFile, and RCFile. Tables can be managed or external, partitioned or not. The full list, and how each format is read, is in Supported tables and formats.

Are text tables read the way Hive reads them?

Yes. Qualytics uses the SerDe settings the metastore records for the table, and Hive's defaults where a setting is absent, so a LazySimpleSerDe table with no field delimiter is split on Ctrl-A and \N is read as null. A text table whose settings cannot be carried over exactly, for example one with an escape character or footer lines, is read through Hive's own SerDe instead, so the results still match Hive.

Why is one of my tables reported as unsupported?

Because reading it directly would return something different from what Hive reports, so Qualytics declines it instead. That covers transactional (ACID) tables, views, tables backed by a storage handler such as HBase or Kudu, Delta, Hudi, Iceberg, and Paimon tables registered in Hive, tables registered with SymlinkTextInputFormat, tables whose input format or SerDe is not available to Qualytics, and tables created with a USING data source other than Parquet, ORC, CSV, or JSON. The reason is reported with each table. Read those through the Hive connector.

Does one unsupported table break the sync?

No. Each declined table is reported on its own, and every other table in the database syncs normally.

Can Qualytics write to my Hive cluster?

No. The connector is read-only, so it cannot be used as an enrichment datastore. Link a separate enrichment datastore as its destination for the anomalies and metadata Qualytics produces.

Is a Computed Table query written in HiveQL or Spark SQL?

Spark SQL. Hive Native reads the table files itself, so Qualytics runs a Computed Table query instead of sending it to HiveServer2. Most HiveQL reads the same in Spark SQL, but Hive UDFs registered on your cluster and Hive session settings are not available. If a query depends on them, create the Computed Table on a Hive datastore instead, where it is written in HiveQL. See SQL Dialects per Connector.


Large tables

Which rows does a profile or scan with a record limit read?

The newest partitions first. On a partitioned table that holds more rows than the limit, Qualytics reads whole partitions from the newest back until it has enough rows, so the operation checks the most recent data rather than a sample spread across the table. See Newest partitions first.

Why does the total record count differ from a count in Hive?

Profiles do not count every row, and scans count them only when they run a Min Partition Size or Max Partition Size check. The total comes from the rows the operation read, or from Hive's table statistics, which can be out of date. See Row counts.

Can an incremental scan use a partition column stored as text?

Yes, when it is the table's first partition column. A column of dates as text, such as 20261112, is found by Sync and set as the Last Modified field when the table has no incremental field yet, and a column of batch numbers as text can be set as the Batch Value field. See Incremental scans on a text partition column.

Is there a limit on how many batches an incremental scan reads?

No. An incremental scan reads every batch after the last one it scanned, in that one scan. When more than 1,000 batches are new, it fetches the details of every partition from the metastore first, as a Full scan does, so it takes longer to start. See Incremental scans on a text partition column.

Does a scan with a record limit skip records in later incremental scans?

It can. When the incremental field is the first partition column holding dates or batch numbers as text, the next Incremental scan starts after the newest partition, so older partitions the limit left out are not read later. For other incremental fields, records the limit left out can be skipped too, depending on their values and on whether the scan counted its records. Run a Full scan without a record limit to check what was left out. See Record limits and incremental scans.


Connecting

Why is there no Catalog field?

Connectors such as Athena ask for a Catalog, the level above the database. Hive has no such level, only databases and tables, so there is nothing to name. Qualytics works one out from the metastore address, which also makes it impossible for two datastores to collide while pointing at different clusters.

Do I need the Warehouse Directory?

Not for reading. Qualytics takes each table's location from the metastore rather than from a warehouse root, so the field is optional.

My tables are in Amazon S3 rather than HDFS. Does that work?

Yes, and it needs less network access: only the metastore and the Key Distribution Center. Qualytics reads object storage with the credentials your deployment is already configured with.

The connection test passed and the sync worked, but the profile failed. Why?

Reading table information never touches storage, so a datastore whose storage is unreachable connects cleanly, syncs cleanly, and then fails on the first operation that reads data. Allow the NameNode and DataNode ports, and confirm that every hostname recorded in the table locations resolves from your Qualytics deployment. See Troubleshooting.


Kerberos

Is Kerberos the only option?

Yes. The metastore protocol has no username and password authentication of its own, so there is no other credential to collect. To connect to Hive as a specific named user, use the Hive connector.

Do I still need deployment-wide Kerberos configuration?

No. Each connection carries its own Client Principal and keytab, so reaching a Kerberized cluster is a change to the connection rather than to your deployment. Helm chart 2026.9.1 and later no longer offer a deployment-wide Kerberos setting, so upload a keytab on every connection. Older self-hosted deployments that still have one should move their connections to their own keytabs before upgrading. See Connections without a keytab.

How do Client Principal and Metastore Service Principal differ?

Client Principal is the identity in your keytab, for example qualytics@EXAMPLE.COM. Metastore Service Principal identifies the metastore, for example hive/_HOST@EXAMPLE.COM. If omitted, the metastore principal defaults to hive/_HOST@ followed by the login realm. _HOST is supported only in the metastore principal. See Authentication.

Can I connect several Kerberized clusters?

Yes, each with its own identity and its own krb5.conf. The one arrangement that cannot work is two clusters using the same realm name with different Key Distribution Centers.

Do I need to renew tickets?

No. Qualytics renews them from the keytab on its own. The only thing to do by hand is upload a new keytab after a credential change.

My keytab upload was rejected.

The file has to be the keytab itself and has to be named with a .keytab extension. Uploading a krb5.conf into the Keytab field produces the same rejection.