BigQuery Connector

Google BigQuery is a serverless, fully managed data warehouse on Google Cloud that runs standard SQL over data stored in its columnar storage and in external sources. Teams use it to centralize analytics workloads without provisioning or sizing any infrastructure.

The Qualytics BigQuery connector connects through the BigQuery JDBC driver using a Google Cloud service account. Point it at a project and its datasets, and Qualytics will profile tables, run scheduled scans, and surface record- and schema-level anomalies alongside the rest of your governed sources.

Notes

Enrichment support. BigQuery can also host an enrichment datastore, so Qualytics can write anomalies, remediation tables, and record enrichment back to a dataset you choose.

Temp dataset. The connection accepts an optional temporary dataset where the driver stores large query results and temporary tables when BigQuery is a source datastore. Creating one with a short table expiration keeps storage costs down; see Create a BigQuery Temp Dataset for the setup.

Large results. The driver retrieves large result sets through the BigQuery Storage Read API, which is why the service account needs the read session role listed on the Permissions page.

Cost model. BigQuery on-demand pricing bills per data scanned, so profile and scan operations directly affect your Google Cloud bill. Tables that are partitioned and clustered scan less data than unpartitioned ones; preferring these for the tables Qualytics monitors keeps scan cost predictable.

Deep Dive

How-tos