# Hive Native Best Practices

Source: https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/best-practices/

Recommendations for getting reliable results and predictable run times from a Hive Native datastore.

## Plan the connection

**Confirm storage reachability before the first profile, not after.** A datastore whose metastore is reachable but whose storage is not will connect, sync, and list every table without complaint, then fail on the first operation that reads records. Checking the NameNode and DataNode ports and the hostname resolution up front turns a confusing failure into a five-minute setup step. See [Permissions](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/permissions/).

**Use one connection per cluster and reuse it.** Every datastore that reads from the same metastore should share one saved connection. The credentials then live in one place, so changing a keytab is a single edit rather than one per database. Add further databases with [Add with an Existing Connection](https://userguide.qualytics.io/source-datastore/datastore/native/hive/how-tos/add-existing-connection/).

**Leave Warehouse Directory empty unless you have a reason not to.** Qualytics takes each table's location from the metastore, so the warehouse root is not consulted when reading. Filling it in suggests to the next reader that it matters.

---

## Choose the right connector per table

**Send file-based tables through Hive Native and only what it declines through Hive.** The native path is the faster one, and it reads Parquet, ORC, delimited text, JSON, Avro, and other SerDe-based tables. What it declines is mostly views, transactional tables, tables backed by a storage handler, and Delta, Hudi, Iceberg, and Paimon tables registered in Hive. The full list is in [Supported tables and formats](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/introduction/#supported-tables-and-formats). Pointing both connectors at the same cluster, and sending only those tables through the [Hive](https://userguide.qualytics.io/source-datastore/datastore/jdbc/hive/) connector, gets you the speed for most of the database and full coverage for the rest.

**Do not treat a declined table as a failure to work around.** A refusal means reading those files directly would disagree with what Hive itself returns. Rewriting the table to satisfy the native path is rarely worth it; reading it through the [Hive](https://userguide.qualytics.io/source-datastore/datastore/jdbc/hive/) connector is.

---

## Keep operations fast

**Keep Hive's table statistics current.** Profiles, and scans without a Min Partition Size or Max Partition Size check, do not count a table's rows. The total they report comes from the rows they read or from the row count in Hive's table statistics. Running Hive's own statistics collection after a significant load keeps that total close to the real count; statistics left out of date make it drift away from it.

**Let incremental scans follow a text partition column.** When a table is partitioned by a date or a batch number stored as text, keep that column, the table's first partition column, as its incremental field. An incremental scan then reads only the new partitions, while a data column as the incremental field makes every incremental scan search all the partitions for new rows. Keep stray values such as `test` out of that column: they are left out of incremental scans, and when they make up half or more of the values, the column is not recognized as a date column at all. See [Incremental scans on a text partition column](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/how-it-works/#incremental-scans-on-a-text-partition-column).

**Narrow the datastore to the databases you actually monitor.** A datastore over a database with a very large number of partitions spends real time on partition metadata. Selecting only the databases you intend to profile keeps sync and scan times proportional to what you care about.

**Filter scans on partition columns.** A scan whose conditions name the partition columns lets Qualytics ask the metastore for just those partitions rather than fetching the whole partition list, which is the difference between reading one day and enumerating every day the table holds.

---

## Keep Kerberos manageable

**Put the credentials on the connection.** Uploading the keytab and `krb5.conf` on the connection means reaching a secured cluster is a change to the connection rather than to your deployment. It also makes a second cluster a matter of adding a connection.

**Give each cluster a distinct realm name.** Two clusters sharing a realm name with different Key Distribution Centers cannot coexist on one deployment, and the second connection is refused. This is worth settling before it becomes an obstacle.

**Keep one identity per purpose.** The keytab identifies the account Qualytics reads as, and every table read is checked against it. An identity scoped to the databases Qualytics monitors makes an access problem obvious in your cluster's own audit trail rather than hidden behind a shared service account.

**Upload the new keytab as soon as a credential changes.** Qualytics renews tickets from the keytab on its own, so there is nothing to run on a schedule, but a keytab from before a change stops working the moment the old key is retired.

---

## Keep the data readable

**Keep column-name case consistent.** Hive records column names in lower case while the data files keep the case they were written with. Where they disagree, Qualytics refuses the scan rather than returning a column full of empty values. Writing files with the names Hive records avoids the situation entirely.

**Prefer partitioned, columnar tables for the data you monitor.** Partitioning is what lets a scan read a slice instead of the whole table, and Parquet and ORC are read straight from their files, which is the most direct path the connector has. Text, JSON, and Avro tables are read too, so there is no need to convert them just to monitor them, but a table that is both partitioned and columnar is the case this connector is built for.

---

## See Also

- **Permissions**

  The access your Hive cluster must grant, plus the Qualytics user role and team permission needed to manage the datastore.

  [Permissions](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/permissions/)
- **How It Works**

  One read from start to finish: the services Qualytics reaches, on which ports, and where your cluster's policies apply.

  [How It Works](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/how-it-works/)
- **Authentication**

  Kerberos on this connector: the service principal, the keytab Qualytics presents, and the `krb5.conf` that reaches the realm.

  [Authentication](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/authentication/)
- **Examples**

  Worked scenarios: a partitioned Parquet warehouse, a database of mixed formats, object storage, and two Kerberized clusters.

  [Examples](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/examples/)

*Last updated: November 13, 2026.*
