# Hive Introduction to the Hive Native Connector

Source: https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/introduction/

Apache Hive organizes data held in HDFS or cloud object storage into databases and tables, and records in its metastore where every table and partition lives. Most Hive environments expose that data through HiveServer2, which compiles each query, runs it on the cluster, and returns the rows.

The Qualytics Hive Native connector takes a shorter path. It reads table information from the metastore and then reads the table's data files directly, leaving the cluster's query engine out of the read path, which is considerably faster on large tables. Partitions come from the metastore rather than from listing storage, so Qualytics profiles tables, runs scheduled scans, and surfaces record- and schema-level anomalies on results that match what Hive itself reports.

---

> **Note: Source datastores only**
>
> Hive Native is read-only, so it cannot be used as an enrichment datastore. Link a separate enrichment datastore as its destination to store the anomalies and metadata Qualytics produces, either [during creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/) or [afterwards](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-enrichment/).

## Notes

**Choosing between the two Hive connectors.** Qualytics keeps both, and they are good at different things. Use Hive Native for the file-based tables that make up most large Hive environments, whether stored as Parquet, ORC, delimited text, JSON, or Avro, and the [Hive](https://userguide.qualytics.io/source-datastore/datastore/jdbc/hive/) connector for the tables the native path declines to read. Both can be connected against the same cluster at the same time.

|  | Hive Native | Hive |
| --- | --- | --- |
| **How it reads data** | Reads the table's data files directly | Runs queries through HiveServer2 |
| **Speed on large tables** | Considerably faster | Limited by HiveServer2 |
| **Table formats** | Parquet, ORC, delimited text, JSON, Avro, and other formats Hive reads through a SerDe | Any format Hive can query |
| **Views** | Not supported | Supported |
| **Transactional (ACID) tables** | Not supported | Supported |
| **Connects to** | The Hive metastore | HiveServer2 |
| **Authentication** | Kerberos | Kerberos, or user and password |
| **Computed Table queries** | Spark SQL, run by Qualytics | HiveQL, run by HiveServer2 |

**Supported tables and formats.** The connector reads managed and external tables, partitioned or not, in these formats:

| Stored as | How it is read |
| --- | --- |
| Parquet and ORC | Directly from the data files |
| Delimited text (`LazySimpleSerDe`, `OpenCSVSerde`) and JSON (the Hive, HCatalog, and OpenX JSON SerDes) | Directly from the data files, using the table's SerDe settings |
| Avro tables created by Hive, `RegexSerDe`, SequenceFile, RCFile, and other SerDes available to Qualytics | Through Hive's own SerDe |

Text and JSON tables are read with the SerDe settings the metastore records for them, including Hive's defaults where a setting is absent. A `LazySimpleSerDe` table with no field delimiter set is split on Ctrl-A, as Hive does, not on commas, and `\N` is treated as the null marker. When a text table uses a setting that cannot be carried over exactly, such as an escape character, a line delimiter other than a newline, more than one header line, footer lines, or complex columns, Qualytics reads it through Hive's own SerDe instead, so the results still agree with what Hive returns.

Anything else is declined rather than read incorrectly, and each declined table is reported on its own, with the reason, while the rest of the database syncs normally. Use the [Hive](https://userguide.qualytics.io/source-datastore/datastore/jdbc/hive/) connector for the tables on this list.

| Declined | Why |
| --- | --- |
| Transactional (ACID) tables | Reading the files directly would ignore delete records and return rows Hive itself would not return |
| Views | The native path reads tables, not query definitions |
| Delta, Hudi, Iceberg, and Paimon tables registered in Hive | These formats decide which files are current, so their own reader is required |
| Tables backed by a storage handler, such as HBase or Kudu | Their data is only reachable through the handler |
| Tables registered with `SymlinkTextInputFormat` | Their files list other files rather than holding records |
| Tables whose input format or SerDe is not available to Qualytics | The table cannot be read the way Hive reads it |
| Tables created with a `USING` data source other than Parquet, ORC, CSV, or JSON, such as an Avro table created with `USING avro` | They are registered for the reader of that data source, which the native path does not use |

**What the form does not ask for.** There is no **Catalog** field of the kind Athena asks for, because Hive has no level above the database, only databases and tables. Qualytics works the rest out from the metastore address, which also makes it impossible for two datastores to collide while pointing at different clusters. There is no question about reaching storage by hostname either, because that depends on where your Qualytics deployment runs rather than on the datastore, so Qualytics handles it.

**Very large partitioned tables.** A preview, and a profile or scan with a record limit, read the newest partitions first instead of a sample spread across the whole table. An incremental scan can follow a partition column that holds dates or batch numbers as text, such as `20261112` or `2026-11-12`, and read only the partitions added since the last scan. See [Large partitioned tables](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/how-it-works/#large-partitioned-tables).

**Row counts and column names.** Profiles do not count the rows of a table, and scans count them only when they run a Min Partition Size or Max Partition Size check, because counting a very large table can take longer than the operation itself. The total an operation reports comes from the rows it read or from the row count in Hive's table statistics, so it can differ from an exact count when those statistics are out of date. Hive also records column names in lower case while the data files keep the case they were written with, so Qualytics confirms the two agree before a scan reads any data and refuses the scan if they differ, rather than returning a column full of empty values.

---

## Deep Dive

- **How It Works**

  One read from start to finish: the services Qualytics reaches, on which ports, and where your cluster's policies apply.

  [How It Works](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/how-it-works/)
- **Authentication**

  Kerberos on this connector: the service principal, the keytab Qualytics presents, and the `krb5.conf` that reaches the realm.

  [Authentication](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/authentication/)
- **Examples**

  Worked scenarios: a partitioned Parquet warehouse, a database of mixed formats, object storage, and two Kerberized clusters.

  [Examples](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/examples/)
- **Best Practices**

  Recommendations for planning the connection, splitting tables between the two Hive connectors, and keeping operations fast.

  [Best Practices](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/best-practices/)
- **Permissions**

  The access your Hive cluster must grant, plus the Qualytics user role and team permission needed to manage the datastore.

  [Permissions](https://userguide.qualytics.io/source-datastore/datastore/native/hive/deep-dive/permissions/)

---

## How-tos

- **Add with New Connection**

  Add Hive Native as a source datastore and create its connection at the same time, field by field.

  [Add with New Connection](https://userguide.qualytics.io/source-datastore/datastore/native/hive/how-tos/add-new-connection/)
- **Add with Existing Connection**

  Add another database from a cluster you already connected, reusing a saved Hive Native connection.

  [Add with Existing Connection](https://userguide.qualytics.io/source-datastore/datastore/native/hive/how-tos/add-existing-connection/)

---

## Reference

- **Troubleshooting**

  Known problems and how to resolve them, grouped by the step that failed: the metastore, Kerberos, storage, or a table.

  [Troubleshooting](https://userguide.qualytics.io/source-datastore/datastore/native/hive/troubleshooting/)
- **API**

  Create, test, read, update, and delete Hive Native datastores over the API, including bulk creation from several databases.

  [API](https://userguide.qualytics.io/source-datastore/datastore/native/hive/api/)
- **FAQ**

  Short answers to the questions that come up most often about the two Hive connectors, formats, and Kerberos.

  [FAQ](https://userguide.qualytics.io/source-datastore/datastore/native/hive/faq/)

*Last updated: November 13, 2026.*
