Hive Native Examples

Real scenarios showing what Hive Native does with a cluster arranged in different ways. Pick a tab to see what to expect from the Sync, the Profile, and the Scan in each situation.

Partitioned WarehouseMixed FormatsObject StorageExternal PartitionsTwo Clusters

Reading a partitioned Parquet warehouse

Context. A team keeps fact and dimension tables in a Hive database called sales, written as Parquet and partitioned by day on HDFS. The metastore is Kerberized. They add a Hive Native datastore pointing at thrift://hive-metastore.internal:9083, leaving Warehouse Directory empty, and select sales as the database.

What happens. The Sync lists every table in sales and records each schema from the metastore. No storage is read at this point, so it finishes quickly even though the warehouse holds thousands of partitions. The first Profile is where storage is touched. When a Scan filters to a single day, Qualytics asks the metastore for the partitions matching that day and reads only those files, instead of listing the table directory and discarding the rest.

Why it works. Table information and data live behind different services, and Hive Native only pays for the one it needs. Leaving Warehouse Directory empty is correct here: each table's location comes from the metastore, so the warehouse root is never consulted.

A database where only some tables can be read natively

Context. A realistic database rarely holds Parquet alone. This one has six tables, and the team wants to know what they will get before they connect it.

Table	Stored as	Result
`orders`	Parquet, partitioned	Read
`customers`	ORC	Read
`order_audit`	Parquet, transactional (ACID)	Declined
`v_active_customers`	View	Declined
`raw_events`	TEXTFILE	Declined
`sessions_delta`	Delta, registered in Hive	Declined

What happens. The Sync succeeds. orders and customers become containers to profile and scan. The other four are reported individually as unsupported, and none of them holds up the rest of the database. The team then connects the same cluster with the Hive connector and points a second datastore at the same database, monitoring the Parquet and ORC tables through Hive Native and the remaining four through HiveServer2.

Why it works. The four are refused because reading their files directly would disagree with what Hive itself returns. order_audit has delete records the files alone do not reflect, sessions_delta has a transaction log that decides which files are current, and the other two need a reader the native path does not use. Running both connectors against one cluster is a supported arrangement, not a workaround.

Tables held in Amazon S3 rather than HDFS

Context. The same Hive metastore, except the table locations it records are s3a:// paths. The team expects to configure storage credentials somewhere in the connection.

What happens. Nothing in the form changes. The Metastore URI and the Kerberos fields are exactly as they would be on HDFS, because Kerberos secures the metastore rather than the storage. What changes is the network access: the connection needs the metastore and the Key Distribution Center, and nothing else. There are no NameNode or DataNode ports to allow.

Why it works. Qualytics reads object storage with the credentials your deployment is already configured with, so the connection never asks for them. This is the cheapest arrangement to set up, since the two ports that most often need a firewall change are not involved.

A partition stored outside the table directory

Context. A table was registered with ALTER TABLE ... ADD PARTITION ... LOCATION pointing at a directory outside the table's own path, which is common once older partitions are moved to cheaper storage. The team worries those partitions will be skipped.

What happens. The partition is read. Qualytics builds its file list from the partition list the metastore holds rather than by listing the table directory, so a partition parked elsewhere is included, and a directory the metastore has since dropped is not.

Why it works. Hive does not require partitions to live under the table location, so listing the directory would be wrong in both directions: it would miss relocated partitions and pick up leftovers the metastore no longer knows about. Taking the list from the metastore is what makes the results match what Hive reports.

Two Kerberized clusters on one deployment

Context. A production cluster and an analytics cluster, each Kerberized, each with its own realm. The team wants both monitored from the same Qualytics deployment.

	Cluster A	Cluster B
Metastore URI	`thrift://hive-a.prod.internal:9083`	`thrift://hive-b.analytics.internal:9083`
Service Principal	`hive/_HOST@PROD.EXAMPLE.COM`	`hive/_HOST@ANALYTICS.EXAMPLE.COM`
Keytab	Cluster A's keytab	Cluster B's keytab
krb5.conf	Cluster A's `krb5.conf`	Cluster B's `krb5.conf`

What happens. Two connections, two identities, one deployment. Neither needs deployment-wide Kerberos configuration, because each connection carries its own credentials.

Why it works. Credentials live on the connection, so adding a cluster is adding a connection. The one arrangement that cannot work is both clusters using the realm name EXAMPLE.COM with different Key Distribution Centers: the second connection is refused, because one realm name cannot mean two clusters and refusing is safer than reading from the wrong one.