# BigQuery BigQuery API

Source: https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/how-tos/api/

This page documents BigQuery-specific payload fields for the datastore API. For generic operations (get, list, update, delete, multi-schema bulk), see [Datastore API](https://userguide.qualytics.io/source-datastore/datastore/api/).

> **Tip: Complete API Reference**
>
> For the full interactive API documentation with all request/response schemas, visit the [API docs](https://userguide.qualytics.io/api/docs).

BigQuery authenticates with a single credential: the service account key. You can create a datastore with a new inline connection or reuse an existing connection, and BigQuery supports both source and enrichment datastores. All modes use the same endpoint:

**Endpoint**: `POST /api/datastores`

**Permission**: Manager, plus the **Editor** team permission on at least one of the assigned teams (the **Public** team when `teams` is left out). Admins are exempt.

---

## UI ↔ API field mapping

The API names a few fields differently from the UI. The mapping below avoids the most common copy-paste mistake (sending the dataset name where the project belongs).

| UI form field | API field | BigQuery concept | Example |
| --- | --- | --- | --- |
| **Connection > Connection Name** | `connection.name` | Label for the saved connection | `bigquery_production` |
| **Connection > Service Account Key** | `connection.password` | The contents of the service account key JSON file, passed as a string | `{"type":"service_account",...}` |
| **Connection > Billing Project ID** | `connection.parameters.billing_project_id` | Project billed for query processing; blank uses the datastore's `database` | `acme-billing` |
| **Connection > Temp Dataset ID** | `connection.parameters.temp_dataset` | Dataset for intermediate query results in the datastore's data project | `qualytics_temp` |
| **Datastore > Project ID** | `database` | Google Cloud project containing the data | `acme-analytics` |
| **Datastore > Dataset ID** | `schema` | Dataset inside the project | `sales_curated` |

## Connection Fields

| Property | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `database` | string | Yes |  | The data project's **Project ID**. Keep this set to the data project even when query billing uses another project. |
| `schema` | string | Yes |  | The **Dataset ID** inside the project (the value shown as **Dataset ID** in the UI). |
| `password` | string | Yes |  | The contents of the service account key JSON file. The key file is itself JSON, so escape it as a JSON string when embedding it in the request body. |
| `parameters.billing_project_id` | string | No |  | Optional billing project override on the connection. Omitted or empty uses each datastore's `database` for billing. |
| `parameters.temp_dataset` | string | No |  | Dataset name for intermediate results in the datastore's data project (`database`), even when `billing_project_id` is set. It must have the required write permissions and match the queried data's location. |

`password` and `parameters` belong inside `connection` when creating an inline connection. When reusing `connection_id`, its saved billing project and temp dataset settings are reused too. The service account key's `project_id` does not select billing.

> **Note: About trigger_sync**
>
> The `trigger_sync: true` flag in the examples below tells Qualytics to immediately sync metadata after creating the datastore (equivalent to ticking the **Initiate Sync** checkbox in the UI). Set it to `false` if you want to defer the first sync.

---

## Create a Source Datastore

**Tab: New Connection**

This example reads `acme-analytics.sales_curated`, bills query processing to `acme-billing`, and uses the pre-created `acme-analytics.qualytics_temp` dataset for intermediate results. Grant the [required permissions](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/permissions/) before sending the request. To bill the data project instead, omit `billing_project_id`; the temp dataset stays where it is.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "BigQuery Production",
>     "connection": {
>       "name": "bigquery_production",
>       "type": "bigquery",
>       "password": "<contents of the service account key JSON, escaped as a string>",
>       "parameters": {
>         "billing_project_id": "acme-billing",
>         "temp_dataset": "qualytics_temp"
>       }
>     },
>     "database": "acme-analytics",
>     "schema": "sales_curated",
>     "teams": ["Data Platform"],
>     "trigger_sync": true
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 42,
>   "name": "BigQuery Production",
>   "type": "bigquery",
>   "store_type": "jdbc",
>   "database": "acme-analytics",
>   "schema": "sales_curated",
>   "connected": true
> }
> ```

**Tab: Existing Connection**

If you have already created a BigQuery connection (via this API or the UI), reuse it by passing its `connection_id`. The credentials, billing project, and temp dataset are reused from the existing connection: you only need to specify datastore-level fields like `database`, `schema`, and `teams`. A saved `billing_project_id` applies to every datastore using that connection; when it is blank, each datastore's `database` is billed.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "BigQuery Finance",
>     "connection_id": 7,
>     "database": "acme-analytics",
>     "schema": "finance_curated",
>     "teams": ["Finance Team"],
>     "trigger_sync": true
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 43,
>   "name": "BigQuery Finance",
>   "type": "bigquery",
>   "store_type": "jdbc",
>   "database": "acme-analytics",
>   "schema": "finance_curated",
>   "connected": true
> }
> ```

---

## Create an Enrichment Datastore

BigQuery also supports enrichment datastores. The payload is the same as for a source datastore, plus `"enrichment_only": true`. Point `schema` at the dataset Qualytics should write enrichment outputs to; the service account needs the read-write roles listed on the [Permissions](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/permissions/) page.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "BigQuery Enrichment",
>     "connection_id": 7,
>     "database": "acme-analytics",
>     "schema": "qualytics_enrichment",
>     "teams": ["Data Platform"],
>     "enrichment_only": true
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 44,
>   "name": "BigQuery Enrichment",
>   "type": "bigquery",
>   "store_type": "jdbc",
>   "database": "acme-analytics",
>   "schema": "qualytics_enrichment",
>   "connected": true
> }
> ```

> **Warning: Field name**
>
> The flag is `enrichment_only`, not `enrich_only`. The API silently ignores unknown fields, so a misspelled flag creates a source datastore instead of an enrichment datastore.

To link an enrichment destination to a source datastore, see [Datastore API › Link Enrichment Destination](https://userguide.qualytics.io/source-datastore/datastore/api/#link-an-enrichment-destination).

---

## Other Operations

For get, list, update, delete, test connection, multi-schema bulk creation, and end-to-end automation examples, see the [Datastore API](https://userguide.qualytics.io/source-datastore/datastore/api/) reference.

## Change or Clear the Billing Project

**Endpoint**: `PUT /api/connections/{id}`

**Permission**: Manager, with permission to edit the connection. See [Manage Connections](https://userguide.qualytics.io/settings/connections/manage-connections/) for access requirements.

Changing a connection affects every datastore that uses it. Keep their `database` and `schema` values unchanged. Prepare the query permissions in the new billing project before updating the connection. The temp dataset stays in each datastore's data project.

Read the connection with `GET /api/connections/{id}` first. The `parameters` object **replaces the saved object**, so include every parameter you want to retain. The following example assumes the connection is named `bigquery_production` and has only the two parameters shown. Omit `password` to retain the saved service account key.

> **Example: Set a separate billing project**
>
> ```bash
> curl -X PUT "https://your-instance.qualytics.io/api/connections/7" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "bigquery_production",
>     "type": "bigquery",
>     "parameters": {
>       "billing_project_id": "acme-billing",
>       "temp_dataset": "qualytics_temp"
>     }
>   }'
> ```
>
> A successful update returns **200 OK** with the updated connection. Queries now run in `acme-billing`, and the temp dataset still refers to `<database>.qualytics_temp` for each datastore.

To test the proposed settings without saving them, send the same body to `POST /api/connections/7/test`. Success returns **204 No Content**.

> **Example: Restore billing to each datastore's data project**
>
> ```bash
> curl -X PUT "https://your-instance.qualytics.io/api/connections/7" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "bigquery_production",
>     "type": "bigquery",
>     "parameters": {
>       "billing_project_id": "",
>       "temp_dataset": "qualytics_temp"
>     }
>   }'
> ```
>
> A successful update returns **200 OK**. Query billing now uses each datastore's `database`, so the query permissions must exist in those projects. The temp dataset was already there.

*Last updated: November 3, 2026.*
