Skip to content

Create a BigQuery Temp Dataset

The BigQuery connection accepts an optional Temp Dataset ID for intermediate query results. This dataset is separate from your source data and any enrichment destination. Creating it with a short table expiration keeps intermediate data from accumulating.

Create it in the connection's Billing Project ID, or in the datastore's Project ID when billing is left blank. A temp dataset in the data project is not used when a separate billing project is selected.

Before you start

Ask a Google Cloud administrator to create the dataset and grant the service account roles/bigquery.dataEditor on it. If the driver must create a missing dataset instead, the account needs bigquery.datasets.create on the project. Query permissions alone do not provide that access. See Permissions.

Steps

Step 1: Navigate to the BigQuery console within your Google Cloud Platform (GCP) account.

step-1-bigquery-console

Step 2: Find the billing project in the Explorer, open its actions menu, and click Create dataset. If Billing Project ID is blank, choose the datastore's data project.

step-2-create-dataset-menu

Step 3: Fill in the details to create the new dataset.

Info

  • Dataset ID: choose a name for the temp dataset. The example below uses Qualytics_12.
  • Dataset Location: select the same location as the data Qualytics will query. For example, a source dataset in the US multi-region requires a temp dataset in US; a different region is not interchangeable.
  • Default Table Expiration: set the expiration to 1 day to ensure any table created in this dataset is automatically deleted one day after its creation.

step-3-dataset-details

Step 4: Click the Create Dataset button to apply the configuration and create the dataset.

step-4-create-dataset-button

Step 5: Grant the Qualytics service account roles/bigquery.dataEditor on the created dataset, then open Dataset info. Enter only the dataset name (Qualytics_12 in this example) in the connection's Temp Dataset ID field, without the project prefix shown by Google. Test the connection with the selected data project and dataset.

step-5-dataset-id

Where the field lives

The Temp Dataset ID field is part of the connection's properties; see the Add Source Datastore field reference. Changing or clearing Billing Project ID changes where that temp dataset name is resolved, so prepare the dataset and permissions in the new project before saving the connection.