Hive Native API

Everything you can do with a Hive Native datastore over the API: create one with a new or an existing connection, test the connection, read and list datastores, update or delete one, and create several at once from the databases of a single metastore.

Complete API Reference

For the full interactive API documentation with all request and response schemas, visit the API docs.

Endpoint: POST /api/datastores

Permission: Manager

Source datastores only

Hive Native is read-only and cannot be used as an enrichment datastore. To link a separate enrichment datastore as the Hive Native source's destination, see Datastore API › Link Enrichment Destination.

UI ↔ API field mapping

The API names several fields differently from the connection form, and the two Kerberos files travel in places worth knowing about before you build a payload.

UI form field	API field	Notes
Connection > Metastore URI	`connection.parameters["hive.metastore.uris"]`	`thrift://metastore-host:9083`, comma separated for a highly available metastore
Connection > Warehouse Directory	`connection.parameters["hive.metastore.warehouse.dir"]`	Optional, and not needed for reading
Connection > Type	`connection.parameters.authentication_type`	`KERBEROS` is the only accepted value
Connection > Client Principal	`connection.username`	The login identity in the keytab, for example `qualytics@EXAMPLE.COM`
Connection > Metastore Service Principal	`connection.parameters["hive.metastore.kerberos.principal"]`	Optional service identity; defaults to `hive/_HOST@` followed by the Kerberos login realm
Connection > Keytab	`connection.password`	The keytab file, base64 encoded
Connection > krb5.conf	`connection.parameters.krb5_base64`	The file's contents
Connection > RPC Protection	`connection.parameters["hadoop.rpc.protection"]`	`authentication`, `integrity`, or `privacy`; omitted means `authentication`
Connection > hdfs-site.xml	`connection.parameters.hdfs_site_xml`	The file's contents, as XML or base64 encoded
Location > Database	`schema`	The Hive database to monitor

Why the keytab travels as password

A keytab is a password equivalent, and password is the field Qualytics stores encrypted. Sending it there is what keeps it out of the plain-text connection parameters. It is never returned on a read: responses report only whether a keytab is stored.

Create a Hive Native Datastore

Both flows use the same endpoint and differ only in whether the connection travels inline or by reference.

New ConnectionExisting Connection

Pass the metastore address and the Kerberos credentials inline. Qualytics saves the connection and creates the datastore in a single call, so the connection is then available to reuse.

The keytab must be base64 encoded, because it is a binary file; the krb5.conf is text and is sent as-is. Both are optional: omit them to fall back to the Kerberos credentials configured on the deployment.

If the new connection has no keytab, Qualytics uses the deployment's configured principal and keytab. Supply the deployment's login principal as the required username value. This value is ignored for login and is not checked against the deployment identity; changing it alone does not select a different login. The deployment principal's realm supplies the default metastore service realm.

Connection fields

Property	Type	Required	Default	Description
`type`	string	Yes		`hive_native`.
`name`	string	Yes		A label for the saved connection.
`parameters["hive.metastore.uris"]`	string	Yes		The metastore address, for example `thrift://metastore-host:9083`.
`parameters["hive.metastore.warehouse.dir"]`	string	No		The warehouse root, for example `/user/hive/warehouse`.
`parameters.authentication_type`	string	No	`KERBEROS`	Filled in as `KERBEROS` when omitted. Any other value is rejected.
`parameters.krb5_base64`	string	No		The contents of the cluster's `krb5.conf`.
`username`	string	Yes		The client principal in the keytab, for example `qualytics@EXAMPLE.COM`. A value of the form `service/_HOST` or `service/_HOST@REALM` is rejected.
`parameters["hive.metastore.kerberos.principal"]`	string	No		The metastore service principal. Defaults to `hive/_HOST@` followed by the login realm; set it explicitly for a different service name or realm.
`password`	string	No		The keytab, base64 encoded.
`parameters["hadoop.rpc.protection"]`	string	No	`authentication`	The protection level of the metastore connection: `authentication`, `integrity`, or `privacy`. Must match the metastore's setting.
`parameters.hdfs_site_xml`	string	No		The cluster's `hdfs-site.xml`, as XML or base64 encoded. Required when table locations use a highly available nameservice. Only its `dfs.*` properties are applied.

Datastore fields

Property	Type	Required	Default	Description
`name`	string	Yes		The datastore name in Qualytics.
`schema`	string	Yes		The Hive database to monitor (shown as Database in the form).
`teams`	array	Yes		The teams that can work with this datastore.
`trigger_sync`	boolean	No	`false`	Ask Qualytics to run the first Sync once the datastore is created, equivalent to ticking Initiate Sync in the form.

No Catalog to send

Connectors such as Athena take a Catalog, the level above the database. Hive has no such level, so there is no field to fill in. Qualytics derives one from the metastore address, which is also what makes it impossible for two datastores to collide while pointing at different clusters.

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/datastores" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Hive Sales",
    "connection": {
      "name": "acme_hive_metastore",
      "type": "hive_native",
      "username": "qualytics@EXAMPLE.COM",
      "password": "<base64-encoded keytab>",
      "parameters": {
        "authentication_type": "KERBEROS",
        "hive.metastore.kerberos.principal": "hive/_HOST@EXAMPLE.COM",
        "hive.metastore.uris": "thrift://metastore-host:9083",
        "hive.metastore.warehouse.dir": "/user/hive/warehouse",
        "krb5_base64": "<krb5.conf contents>",
        "hadoop.rpc.protection": "privacy",
        "hdfs_site_xml": "<hdfs-site.xml contents>"
      }
    },
    "schema": "sales",
    "teams": ["Data Platform"],
    "trigger_sync": true
  }'

Response (200 OK):

{
  "id": 42,
  "name": "Hive Sales",
  "type": "hive_native",
  "store_type": "native",
  "schema": "sales",
  "connected": true
}

Producing the base64 keytab

On macOS or Linux, base64 -i hive.keytab prints the value to paste into password. Send the krb5.conf as plain text.

Reuse a Hive Native connection you already created, through this API or the connection form, by passing its connection_id. The metastore address and the Kerberos credentials come from the saved connection, so you supply only the datastore fields.

Reuse it whenever you are adding another database from the same cluster. The credentials stay in one place, and changing them later is a single edit rather than one per datastore.

Datastore fields

Property	Type	Required	Default	Description
`name`	string	Yes		The datastore name in Qualytics.
`connection_id`	integer	Yes		The ID of the saved Hive Native connection.
`schema`	string	Yes		The Hive database to monitor (shown as Database in the form).
`teams`	array	Yes		The teams that can work with this datastore.
`trigger_sync`	boolean	No	`false`	Ask Qualytics to run the first Sync once the datastore is created.

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/datastores" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Hive Finance",
    "connection_id": 7,
    "schema": "finance",
    "teams": ["Finance Team"],
    "trigger_sync": true
  }'

Response (200 OK):

{
  "id": 43,
  "name": "Hive Finance",
  "type": "hive_native",
  "store_type": "native",
  "schema": "finance",
  "connected": true
}

Run the first Sync

A datastore is only useful once Sync has discovered its tables. Send trigger_sync as true to start it with the datastore, or run a Sync yourself once the request returns.

Test a Connection

Both tests run as the Member role. The second one also needs the Editor team permission on the datastore.

Before savingAn existing datastore

Validates a create payload without persisting anything. Send the same body you would send to create the datastore.

Endpoint: POST /api/datastores/connection

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/datastores/connection" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Hive Sales",
    "connection": {
      "name": "acme_hive_metastore",
      "type": "hive_native",
      "username": "qualytics@EXAMPLE.COM",
      "password": "<base64-encoded keytab>",
      "parameters": {
        "authentication_type": "KERBEROS",
        "hive.metastore.kerberos.principal": "hive/_HOST@EXAMPLE.COM",
        "hive.metastore.uris": "thrift://metastore-host:9083",
        "krb5_base64": "<krb5.conf contents>"
      }
    },
    "schema": "sales",
    "teams": ["Data Platform"]
  }'

Response (200 OK):

{
  "connected": true,
  "message": null
}

When the connection cannot be verified, the response is a 400 Bad Request whose message names the failure. A message on a 200 is a warning, not an error.

Re-checks a saved datastore. Because Hive Native is read-only, the check confirms that Qualytics can read from the metastore.

Endpoint: POST /api/datastores/{id}/connection

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/datastores/42/connection" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response (200 OK):

{
  "connected": true,
  "message": null
}

A datastore that cannot be reached answers "connected": false with the reason in message.

Note

The metastore answering is not the same as the data being readable. A test can pass while the table files are still unreachable, which shows up as a failed profile rather than a failed test. See Troubleshooting.

Get a Datastore

Endpoint: GET /api/datastores/{id}

Permission: Member

Example request and response

Request:

curl -X GET "https://your-instance.qualytics.io/api/datastores/42" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response (200 OK):

{
  "id": 42,
  "name": "Hive Sales",
  "type": "hive_native",
  "store_type": "native",
  "connection_id": 7,
  "schema": "sales",
  "username": "qualytics@EXAMPLE.COM",
  "parameters": {
    "authentication_type": "KERBEROS",
    "hive.metastore.kerberos.principal": "hive/_HOST@EXAMPLE.COM",
    "hive.metastore.uris": "thrift://metastore-host:9083",
    "hive.metastore.warehouse.dir": "/user/hive/warehouse"
  },
  "has_password": true,
  "connected": true,
  "enrichment_only": false,
  "enrichment_prefix": "_hive_sales",
  "teams": [{ "name": "Data Platform" }]
}

The keytab never comes back

has_password reports only whether a keytab is stored on the connection. Its contents are never returned, and neither is the krb5.conf.

List Datastores

Endpoint: GET /api/datastores

Permission: Member

Example request and response

Request:

curl -X GET "https://your-instance.qualytics.io/api/datastores?datastore_type=hive_native&sort_name=asc" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response (200 OK):

{
  "items": [
    { "id": 42, "name": "Hive Sales", "type": "hive_native", "store_type": "native", "connected": true },
    { "id": 43, "name": "Hive Finance", "type": "hive_native", "store_type": "native", "connected": true }
  ],
  "total": 2,
  "page": 1,
  "size": 50,
  "pages": 1
}

Query Parameters

Parameter	Type	Description
`id`	`list[int]`	Filter by datastore ID.
`name`	string	Filter by exact name.
`search`	string	Partial name match, case-insensitive, or an exact ID.
`datastore_type`	`list[string]`	Filter by connector type. `hive_native` returns only Hive Native datastores.
`tag`	`list[string]`	Filter by tag name.
`group`	`list[int]`	Filter by datastore group ID.
`enrichment_only`	boolean	`false` for source datastores (the default). Hive Native datastores are always source datastores.
`sort_name`, `sort_created`, `sort_favorite`, `sort_containers`, `sort_active_anomalies`	string	`asc` or `desc`.

Update a Datastore

Changes the datastore's own settings. Every request must include name, connection_id, enrichment_only, and enrichment_prefix, even when they are not changing, so send their current values from Get a Datastore. Most other parameters you leave out stay as they are, with two exceptions:

tags and teams replace the whole list when sent.
enrichment_source_record_limit, enrichment_remediation_strategy, and high_count_rollup_threshold go back to their defaults (10, none, and 10) whenever they are left out. Send their current values on every update to keep them; the example below sends the values a datastore without a linked enrichment destination has.

Endpoint: PUT /api/datastores/{id}

Permission: Member role, with the Editor team permission on the datastore. Changing teams also takes the Manager or Admin role.

What lives on the connection

The metastore address, the Client Principal, the Metastore Service Principal, the keytab, and the krb5.conf belong to the connection, not to the datastore. Change them through the connection, and the change reaches every datastore that shares it.

Favorites

Marking a datastore as a favorite has its own endpoint: send {"favorite": true} (or false) to PATCH /api/datastores/{id}/favorite, with the Member role. This update ignores a favorite property.

Property	Type	Required	Description
`name`	string	Yes	The datastore name.
`connection_id`	integer	Yes	The Hive Native connection the datastore reads through.
`enrichment_only`	boolean	Yes	Always `false`. Hive Native datastores are source datastores.
`enrichment_prefix`	string	Yes	The prefix for the tables Qualytics writes to the linked enrichment destination. Send the current value to keep it.
`description`	string	No	Free text shown on the datastore.
`schema`	string	No	The Hive database to monitor.
`tags`	`list[string]`	No	Replaces the datastore's tags.
`teams`	`list[string]`	No	Replaces the datastore's teams. Needs the Manager or Admin role.
`enrichment_source_record_limit`, `enrichment_remediation_strategy`, `enrichment_auto_sync`, `high_count_rollup_threshold`	mixed	No	Settings for the enrichment destination linked to this source.

Example request and response

Request:

curl -X PUT "https://your-instance.qualytics.io/api/datastores/42" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Hive Sales (EMEA)",
    "connection_id": 7,
    "enrichment_only": false,
    "enrichment_prefix": "_hive_sales",
    "enrichment_source_record_limit": 10,
    "enrichment_remediation_strategy": "none",
    "high_count_rollup_threshold": 10,
    "schema": "sales_emea",
    "tags": ["hive", "sales"],
    "teams": ["Data Platform", "Sales Ops"]
  }'

Response (200 OK):

{
  "id": 42,
  "name": "Hive Sales (EMEA)",
  "type": "hive_native",
  "store_type": "native",
  "connection_id": 7,
  "schema": "sales_emea",
  "global_tags": [{ "name": "hive" }, { "name": "sales" }],
  "teams": [{ "name": "Data Platform" }, { "name": "Sales Ops" }]
}

Delete a Datastore

Endpoint: DELETE /api/datastores/{id}

Permission: Admin

Example request

curl -X DELETE "https://your-instance.qualytics.io/api/datastores/42" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response: 204 No Content

Warning

Deletion permanently removes the datastore's containers, checks, anomalies, and operation history from Qualytics. Data already written to an enrichment destination stays there and must be removed separately. The connection stays, and so does every other datastore that uses it.

Create Several Datastores at Once

A Hive metastore usually holds more databases than one. The bulk endpoints take a list of databases and create one datastore per entry, sharing the same connection and settings.

Permission: Manager

New ConnectionExisting Connection

Endpoint: POST /api/connections/datastores/bulk

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/bulk" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "connection": {
      "name": "acme_hive_metastore",
      "type": "hive_native",
      "username": "qualytics@EXAMPLE.COM",
      "password": "<base64-encoded keytab>",
      "parameters": {
        "authentication_type": "KERBEROS",
        "hive.metastore.kerberos.principal": "hive/_HOST@EXAMPLE.COM",
        "hive.metastore.uris": "thrift://metastore-host:9083",
        "krb5_base64": "<krb5.conf contents>"
      }
    },
    "schemas": ["sales", "finance", "inventory"],
    "name_template": "hive_{{schema}}",
    "teams": ["Data Platform"],
    "trigger_sync": true
  }'

Response (200 OK):

{
  "created": [101, 102, 103],
  "errors": [],
  "connection_id": 12
}

Endpoint: POST /api/connections/{connection_id}/datastores/bulk

Example request and response

Request:

curl -X POST "https://your-instance.qualytics.io/api/connections/7/datastores/bulk" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "schemas": ["marketing", "logistics"],
    "name_template": "hive_{{schema}}",
    "teams": ["Data Platform"],
    "trigger_sync": true
  }'

Response (200 OK):

{
  "created": [110, 111],
  "errors": [],
  "connection_id": 7
}

Property	Type	Required	Description
`connection`	object	New connection only	The same connection object as in Create a Hive Native Datastore.
`schemas`	`list[string]`	Yes	The Hive databases to create datastores for, one datastore each.
`name_template`	string	No	Naming pattern with `{{schema}}` standing for the database name. Defaults to the connection name followed by the database.
`teams`	`list[string]`	No	Teams assigned to every created datastore.
`tags`	`list[string]`	No	Tags applied to every created datastore.
`group_id`	integer	No	An existing datastore group to place them in.
`description`	string	No	Description applied to every created datastore.
`trigger_sync`	boolean	No	Run the first Sync on each datastore as it is created.

Note

There is no database field to send: Hive has no level above the database, so schemas is the whole address. Creation is not atomic. Databases that succeed are created even if others fail, and the response lists both the created IDs and, per failed database, the reason. It also returns connection_id, the connection the datastores were created on. To retry the failed databases, send them to that connection through the Existing Connection endpoint. When no datastore was created, a New Connection request still returns connection_id if the connection was saved anyway, for example when every database was removed after its enrichment link was refused, so retry on it the same way. When connection_id is null after a New Connection request, the connection was not saved, so resend the whole request. After an Existing Connection request, connection_id is null when no datastore was created; retry on the same connection ID.