Skip to content

BigQuery Troubleshooting

Most BigQuery connection failures fall into three groups: an invalid service account key (authentication), missing BigQuery roles (permissions), or a wrong project or dataset (configuration). Check both the data project and the project billed for queries when they differ.

Common Errors

Error Likely Cause Fix
Access Denied: 403 The service account lacks access to a resource named in the error Check read access on the source datasets, query permissions on the billing project, and write access on any temp dataset; see Permissions
Not found: Dataset The Dataset ID in the connection form does not exist or the service account cannot see it Verify the Dataset ID in the BigQuery Console and ensure the service account has bigquery.dataViewer on the dataset
Not found: Project A project ID is incorrect, or the service account cannot access that project Verify the project named in the error and its permissions; the service account may belong to a different project
The caller does not have bigquery.jobs.create permission The service account lacks permission to run queries in the billing project Grant roles/bigquery.jobUser on Billing Project ID, or on the datastore's Project ID when billing is blank
bigquery.datasets.create permission denied The configured temp dataset does not exist in the billing project and the driver cannot create it Have an administrator create it there and grant dataset write access; see Create a Temp Dataset
bigquery.tables.create permission denied The service account cannot create result tables in the temp dataset Grant roles/bigquery.dataEditor on that dataset in the billing project
bigquery.readsessions.create permission denied Query execution is allowed, but Storage Read API sessions are not Grant roles/bigquery.readSessionUser at project level in the billing project
Unable to parse service account key The JSON key file is malformed or truncated Upload an intact copy of the key, or create a new JSON key in IAM & Admin > Service Accounts

Detailed Notes

Authentication errors

The error Unable to parse service account key means Qualytics could not read the uploaded JSON key file, usually because it was modified or truncated after download. Upload an intact copy or create a new key; Google does not let you download the same private key again.

A key that parses but no longer works fails on Google's side instead. Common causes:

  • Key deleted or disabled: the key was deleted or disabled on the service account in the Google Cloud Console.
  • Missing cross-project access: the service account is valid but lacks access to the data or billing project. An account belonging to another project is supported; grant it the required permissions rather than replacing the key solely because its project differs.

Note

Service account keys do not expire, but they can be deleted or disabled by project administrators. If the key stops working, verify its status in IAM & Admin > Service Accounts > Keys.

Permission errors

The error Access Denied: 403 or The caller does not have bigquery.jobs.create permission means the service account authenticated successfully but lacks the required roles.

Common causes:

  • Missing bigquery.jobUser: the service account cannot run queries without this role. Assign it at project level on the billing project, which defaults to the data project when Billing Project ID is blank.
  • Missing bigquery.dataViewer: the service account cannot read dataset or table metadata.
  • Missing bigquery.readSessionUser: the service account cannot create read sessions for efficient data retrieval via the Storage Read API. A successful small query does not prove that large-result retrieval has the permissions it needs.
  • Dataset-level vs. project-level: both bigquery.jobUser and bigquery.readSessionUser need project-level grants. Source and temp dataset access can be scoped to their datasets.
  • Missing temp dataset access: changing Billing Project ID also changes where Temp Dataset ID is resolved. A dataset and permissions in the previous project do not carry over.

Configuration errors

The error Not found: Dataset or Not found: Project indicates a configuration issue with the Project ID or Dataset ID.

Common causes:

  • Wrong Project ID: Project ID must identify the data project. Use Billing Project ID to change who pays without changing which data is analyzed.
  • Wrong Dataset ID: the Dataset ID was misspelled or does not exist in the specified project.
  • Regional mismatch: the temp dataset is in a different location than the queried data. Both must use the same region or multi-region; see Create a BigQuery Temp Dataset.

Confirm which project is billed

Ask a Google Cloud administrator to inspect the selected billing project's regional INFORMATION_SCHEMA.JOBS_BY_PROJECT view, filtering by the Qualytics service account and the operation's time. Check project_id, user_email, referenced_tables, and total_bytes_billed to distinguish the query's billing project from its source data. Cached queries can report zero billed bytes.

Some short queries return without creating a full BigQuery job. They can still incur query charges and appear in INFORMATION_SCHEMA with a query ID in job_id and a null job_creation_reason.code. An absent entry in ordinary job history does not prove the query was free or billed elsewhere. See Google's JOBS view documentation.

Tip

Investigate in the order BigQuery evaluates requests: first confirm the service account key is valid (authentication errors), then verify the BigQuery roles (permission errors), and finally check the Project ID and Dataset ID (configuration errors).