# BigQuery Add a BigQuery Source Datastore

Source: https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/how-tos/add-source/

A source datastore is a storage location Qualytics connects to so it can profile, scan, and monitor data. Adding BigQuery as a source lets Qualytics query the datasets in your Google Cloud project through the BigQuery JDBC driver and run quality operations on the tables it discovers.

Before you start, review the [BigQuery Permissions](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/permissions/) and the [Authentication](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/authentication/) page for the service account setup.

## Field reference

The **Datastore** page shows the sections below when BigQuery is selected. When reusing an existing connection, the **Connection Properties** and **Secrets Management** sections come already filled in and read-only, so you fill in only the **Location** and the **General** fields. To change a saved connection's credentials or billing project, edit the connection through the [Manage Connections](https://userguide.qualytics.io/settings/connections/manage-connections/) page; edits there apply to every datastore that reuses the connection.

### Connection Properties

These fields define the identity, query billing, and working storage of the connection. They belong to the connection: when reusing an existing connection, they come already filled in and read-only.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Connection Name** | Yes | Text | A label for the saved connection (e.g., `acme_bigquery_warehouse`), so other datastores can reuse it later. |
| **Billing Project ID** | No | Text | Project billed for query processing. Leave blank to use each datastore's **Project ID**. Applies to every datastore using this connection; see [Data and Billing Projects](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/bigquery-connector/#data-and-billing-projects). |
| **Temp Dataset ID** | No | Text | Dataset for intermediate results in the datastore's data project, even when a separate billing project is set. Enter only the dataset name, such as `qualytics_temp`. It needs write access and must match the queried data's location; see [Create a BigQuery Temp Dataset](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/how-tos/create-temp-dataset/). |
| **Service Account Key** | Yes | File | The JSON key file of the service account Qualytics uses. It already contains the account's email and private key. See [Authentication](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/authentication/) for how to create one. Or click **Type a value instead**  and enter a `${key}` reference to the file held in your secrets manager. See [File fields](https://userguide.qualytics.io/source-datastore/datastore/connection/hashicorp-vault/deep-dive/how-it-works/#file-fields). |
| **Max Parallelization** | No | Number | Caps how many queries Qualytics runs at the same time against a datastore reached through this connection, from 1 to 100. Leave it empty for no explicit limit. The cap applies per datastore, so datastores sharing this connection each get their own allowance. Sits in the collapsed **Advanced** section of the form. See [Max Parallelization](https://userguide.qualytics.io/source-datastore/datastore/connection/how-it-works/#max-parallelization). |

### Secrets Management

This group is optional: use it only if you want Qualytics to pull credentials from a secrets manager instead of typing them into the form. Turn on **HashiCorp Vault** to show the fields below. Despite the label, any secrets manager that exposes a compatible REST API works, not only HashiCorp Vault; see [HashiCorp Vault](https://userguide.qualytics.io/source-datastore/datastore/connection/hashicorp-vault/deep-dive/introduction/). It also belongs to the connection: read-only when reusing an existing connection.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Login URL** | Yes | Text | The full URL of the login endpoint (e.g., `https://vault.example.com/v1/auth/approle/login`). With Vault namespaces, add the namespace after `/v1/`, as in `/v1/<namespace>/auth/approle/login`. |
| **Credentials Payload** | Yes | Text | A JSON body containing the credentials Vault expects (e.g., `{"role_id":"...","secret_id":"..."}`). |
| **Token JSONPath** | Yes | Text | The JSONPath that extracts the client token from Vault's response. Defaults to `$.auth.client_token`. |
| **Secret URL** | Yes | Text | The full URL of the secret, including `https://` and the host (e.g., `https://vault.example.com/v1/secret/data/bigquery`). With Vault namespaces, add the namespace after `/v1/`. |
| **Token Header Name** | Yes | Text | The HTTP header name used to send the token. Defaults to `X-Vault-Token`. |
| **Data JSONPath** | Yes | Text | The JSONPath that extracts the key/value pairs from Vault's response. Use `$.data.data` for a KV version 2 secret (its API path contains `/data/`) and `$.data` for KV version 1 or the database secrets engine. The form pre-fills `$.data`. |

> **Note**
>
> Once the secrets manager is configured, reference any secret value in the connection fields using `${key}`. Qualytics resolves the secret at the moment the connection is opened, so changed keys take effect on the next connection. Write the reference with braces (`${key}`, never `$.key`) and match the key name's case. Setup steps, the namespace rule, and a troubleshooting page are on the [HashiCorp Vault](https://userguide.qualytics.io/source-datastore/datastore/connection/hashicorp-vault/deep-dive/introduction/) pages.

### Location

Pick the project and dataset(s) Qualytics should read from. You fill these in on both flows.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Project ID** | Yes | Option | The Google Cloud project that holds your data. It is also billed for queries when **Billing Project ID** is blank. Keep this set to the data project when choosing a separate billing project. Click the refresh icon to fetch the projects visible to the credentials. |
| **Dataset ID** | Yes | Option | One or more datasets inside the selected project. Each dataset you pick becomes its own Qualytics datastore. Click the refresh icon to load the available datasets. |

> **Note: Multiple datasets**
>
> Selecting more than one dataset creates one source datastore per dataset, named from the **Name Template**. See [Multi-Schema Source Datastore Creation](https://userguide.qualytics.io/source-datastore/datastore/multi-schema/overview/) for details.

### General

Common fields for every source datastore, shown below the **Location** section. You fill these in on both flows.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Name Template** | No | Text | Defines the naming pattern for each source datastore being created. Use `{{schema}}` as a placeholder that gets replaced with the actual dataset name (e.g., `bq_{{schema}}` becomes `bq_sales_curated`). Left empty, the datastore is named from the connection name and the dataset. |
| **Group** | No | Option | Organizes your datastores under a shared group in the navigation tree. Select an existing group or create a new one with the **Add New Group** toggle. |
| **Teams** | Yes | Option | Select one or more teams to associate with this source datastore. |
| **Initiate Sync** | No | Checkbox | Automatically sync the datastore to detect containers and fields after creation. |

Below the form, an information banner lists the **Public addresses** and **Private addresses** your datastore connections originate from. Allow the ones that match your network setup through your security groups or firewall rules; see [How Connections Work](https://userguide.qualytics.io/source-datastore/datastore/connection/how-it-works/#finding-the-addresses-to-allow) for where the addresses come from.

## Steps

There are two ways to set up the connection: reuse a connection you already saved (**Existing Connection**) or create a new one from scratch (**New Connection**). The tabs below walk through each option; pick the one you want to follow. Each field is described in the [Field reference](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/how-tos/add-source/#field-reference) above.

**Tab: New Connection**

**Step 1**: Navigate to the **Datastores** page.

**Step 2**: Click the **Add**  button at the top-right corner and choose **Source** .

**Step 3**: The **Datastore** page opens.

**Step 4**: Select **New Connection** next to the **Search**  field.

**Step 5**: Select **BigQuery** from the connector grid. Use the search field to filter connectors by name.

**Step 6**: Fill in the **Connection Properties**: the **Connection Name**, the optional **Billing Project ID** and **Temp Dataset ID**, and the **Service Account Key** JSON file, attached through **Upload File** . If you specify a separate billing project, grant its [query permissions](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/permissions/) first; the temp dataset stays in the data project.

**Step 7**: Optionally, expand **Secrets Management** to retrieve credentials from a secrets manager.

**Step 8**: Fill in the **Location** fields (**Project ID** and **Dataset ID**) and the **General** fields.

**Step 9**: Click **Test connection**. A success message confirms that the connection has been verified.

> **Info**
>
> The **Finish** and **Next** buttons stay disabled until the connection test passes on the current values. If you change a connection field after a successful test, test again. If the test fails, see [Troubleshooting](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/troubleshooting/).

**Step 10**: Click **Finish** to create the datastore.

> **Tip**
>
> To link an enrichment destination so Qualytics can store anomalies and metadata from the first operation, click **Next** instead of **Finish**. See [Link Enrichment on Datastore Creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/).

**Step 11**: A success dialog confirms that your datastore has been added. Click **Go to your datastore** to open its page.

**Tab: Existing Connection**

**Step 1**: Navigate to the **Datastores** page.

**Step 2**: Click the **Add**  button at the top-right corner and choose **Source** .

**Step 3**: The **Datastore** page opens.

**Step 4**: Select **Existing Connection** next to the **Search**  field.

**Step 5**: Select the saved BigQuery connection from the grid. Use the search field to filter connections by name. The **Connection Properties** and **Secrets Management** sections come already filled in and read-only. Check its **Billing Project ID**: a saved value applies to this datastore too; a blank value bills this datastore's **Project ID**.

> **Tip: Start a new connection from this one**
>
> To use the selected connection as a starting point for a brand-new connection instead, click the **Duplicate as a new connection**  button on the selected connection. The form switches to **New Connection** mode with the connection's settings already filled in for you to adjust.

**Step 6**: Fill in the **Location** fields (**Project ID** and **Dataset ID**) and the **General** fields. These are the only fields left to fill in, and among the properties only **Teams** is required.

**Step 7**: Click **Test connection**. A success message confirms that the connection has been verified.

> **Info**
>
> The **Finish** and **Next** buttons stay disabled until the connection test passes on the current values. If the test fails, see [Troubleshooting](https://userguide.qualytics.io/source-datastore/datastore/jdbc/bigquery/troubleshooting/).

**Step 8**: Click **Finish** to create the datastore.

> **Tip**
>
> To link an enrichment destination so Qualytics can store anomalies and metadata from the first operation, click **Next** instead of **Finish**. See [Link Enrichment on Datastore Creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/).

**Step 9**: A success dialog confirms that your datastore has been added. Click **Go to your datastore** to open its page.

*Last updated: November 3, 2026.*
