Amazon SQS

Send messages to Amazon Simple Queue Service for reliable message queuing. Use SignalSmith to deliver model results, audience membership changes, or enriched records to SQS queues for decoupled processing by downstream consumers.

Prerequisites

An Amazon SQS queue (Standard or FIFO)
An IAM user or role with sqs:SendMessage permission on the target queue
AWS access key ID and secret access key for authentication

Authentication

SQS uses AWS Access Keys authentication.

In the AWS IAM Console, create an access key for a user with SQS write permissions
Enter the Access Key ID and Secret Access Key in SignalSmith

Required IAM Permissions

The IAM user needs the following permissions on the target queue:

{
  "Effect": "Allow",
  "Action": [
    "sqs:SendMessage",
    "sqs:SendMessageBatch",
    "sqs:GetQueueAttributes"
  ],
  "Resource": "arn:aws:sqs:REGION:ACCOUNT_ID:QUEUE_NAME"
}

Configuration

Field	Type	Required	Description
AWS Region	Select	Yes	The AWS region where the SQS queue is located. Options include US East (N. Virginia, Ohio), US West (N. California, Oregon), EU (Ireland, Frankfurt, London), and Asia Pacific (Tokyo, Singapore, Sydney)

Target Settings

Field	Type	Required	Description
Queue URL	Text	Yes	The full URL of the SQS queue (e.g., `https://sqs.us-east-1.amazonaws.com/123456789/my-queue`)
FIFO Queue	Toggle	No	Enable if the queue is a FIFO (First-In-First-Out) queue. Default: Off
Message Group ID	Text	Conditional	Required for FIFO queues. Messages with the same group ID are processed in order. Only shown when FIFO Queue is enabled

Supported Operations

Sync Modes: Insert

Audience Sync Modes: Add

Features

Field Mapping: No — SQS publishes the full record as a JSON message body
Schema Introspection: No — SQS queues are schemaless

How It Works

SignalSmith sends each row as a JSON message to the configured SQS queue:

Each row is serialized as a JSON object
Messages are sent to SQS using the SendMessageBatch API for efficiency
For Standard queues, SQS provides at-least-once delivery with best-effort ordering
For FIFO queues, messages are delivered exactly once in strict order within each message group

Standard vs. FIFO Queues

Standard queues: Higher throughput, at-least-once delivery, best-effort ordering
FIFO queues: Exactly-once processing, strict ordering within message groups, lower throughput (300 messages/second without batching)

For FIFO queues, the Message Group ID determines the ordering scope. All messages with the same group ID are delivered in the order they were sent.

Troubleshooting

Access denied

Verify the IAM user has sqs:SendMessage permission on the target queue. Check the queue’s access policy and the IAM user’s policies.

Queue URL invalid

The queue URL must start with https:// and follow the format https://sqs.{region}.amazonaws.com/{account_id}/{queue_name}. Copy the URL directly from the SQS console.

FIFO queue requires message group ID

FIFO queues require a Message Group ID for each message. Enable the FIFO Queue toggle and provide a Message Group ID.

Message too large

SQS has a maximum message size of 256 KB. If individual records exceed this limit, consider reducing the number of fields or using S3 as an intermediate store.

Region mismatch

The selected region must match the region in the queue URL. Verify the queue region in the SQS console.

Google Pub/Sub Google Sheets