BigQuery
Write data back to a Google BigQuery table. Use SignalSmith to materialize model results, audience memberships, or enriched data into BigQuery datasets.
Prerequisites
- A Google Cloud project with BigQuery enabled
- A service account or OAuth credentials with BigQuery Data Editor role
- A BigQuery dataset to write to
Authentication
BigQuery supports two authentication methods:
Service Account
- Create a service account in your GCP project
- Grant it the BigQuery Data Editor role on the target dataset
- Download the JSON key file
- Upload or paste the JSON key in SignalSmith
OAuth
- Click Connect with OAuth in SignalSmith
- Sign in with your Google account
- Authorize BigQuery access
Configuration
| Field | Type | Required | Description |
|---|---|---|---|
| Project ID | Text | Yes | Your Google Cloud project ID |
| Dataset | Text | Yes | The BigQuery dataset to write to |
| Location | Select | No | Dataset location (e.g., US, EU). Default: US |
Target Settings
| Field | Type | Required | Description |
|---|---|---|---|
| Target Table | Text | Yes | The target table name within the dataset |
Supported Operations
Sync Modes: Upsert, Insert, Update, Mirror
Audience Sync Modes: Add, Remove, Mirror, Upsert
Features
- Field Mapping: Yes
- Schema Introspection: Yes — SignalSmith reads table schemas from the BigQuery API
How It Works
SignalSmith uses BigQuery’s DML capabilities for efficient data writing:
- Data is loaded into a staging table using the BigQuery Storage Write API
- A MERGE statement is executed against the target table
- The staging table is cleaned up after the sync
BigQuery identifiers are lowercased to match its case-sensitive quoting behavior.
Troubleshooting
Service account permissions
The service account needs bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create permissions. The BigQuery Data Editor role includes these.
Dataset not found
Verify the dataset exists in the specified project and location. Dataset names are case-sensitive in BigQuery.
Location mismatch
The dataset location must match between your source and destination datasets. Cross-region operations are not supported.
Quota exceeded
BigQuery enforces DML statement quotas. For high-frequency syncs, consider batching or reducing sync frequency.