Google Pub/Sub
Publish messages to Google Cloud Pub/Sub topics for real-time event streaming. Use SignalSmith to send model results, audience membership changes, or enriched records to Pub/Sub for consumption by Cloud Functions, Dataflow pipelines, or other Google Cloud services.
Prerequisites
- A Google Cloud project with the Pub/Sub API enabled
- A Pub/Sub topic to publish messages to
- A service account with Pub/Sub Publisher role on the target topic
Authentication
Pub/Sub uses Service Account authentication.
- In the Google Cloud Console, go to IAM & Admin > Service Accounts
- Create a service account (or use an existing one)
- Grant the Pub/Sub Publisher role (
roles/pubsub.publisher) on the target topic - Create a JSON key for the service account
- Upload or paste the JSON key in SignalSmith
Required Permissions
The service account needs the following role on the target topic:
roles/pubsub.publisher— to publish messages to the topic
Configuration
| Field | Type | Required | Description |
|---|---|---|---|
| GCP Project ID | Text | Yes | The Google Cloud project that contains the Pub/Sub topic (e.g., my-gcp-project) |
Target Settings
| Field | Type | Required | Description |
|---|---|---|---|
| Topic Name | Text | Yes | The Pub/Sub topic to publish messages to (e.g., my-topic) |
Supported Operations
Sync Modes: Insert
Audience Sync Modes: Add
Features
- Field Mapping: No — Pub/Sub publishes the full record as a JSON message
- Schema Introspection: No — Pub/Sub topics are schemaless from the publisher perspective
How It Works
SignalSmith publishes each row as a JSON message to the configured Pub/Sub topic:
- Each row is serialized as a JSON object
- Messages are published to the Pub/Sub topic using the Pub/Sub client library
- Pub/Sub handles message distribution to all subscriptions on the topic
- Messages include metadata attributes for traceability (sync ID, timestamp)
Pub/Sub guarantees at-least-once delivery to subscribers. Messages are retained for up to 7 days if subscribers are offline.
Troubleshooting
Permission denied
Verify the service account has the Pub/Sub Publisher role on the target topic. Check the topic’s IAM policy in the Google Cloud Console.
Topic not found
Ensure the topic name is correct and exists in the specified GCP project. Topic names are case-sensitive. Create the topic in the Pub/Sub section of the Google Cloud Console if it does not exist.
Invalid service account JSON
The JSON key file must be a valid service account key with "type": "service_account". Download a fresh key from the Google Cloud Console if the current one is corrupted.
Quota exceeded
Pub/Sub enforces quotas on publish throughput per project. Check the Pub/Sub quotas page in the Google Cloud Console if you encounter rate limiting. Consider requesting a quota increase for high-volume syncs.
Project ID mismatch
The GCP Project ID must match the project that contains the target topic. Cross-project publishing requires additional IAM configuration.