Loaders API

Loaders are inbound ELT (Extract, Load, Transform) configurations that pull data from external SaaS tools and databases into your data warehouse (Source). They complement Sources by bringing data into the warehouse, while Sources provide read access from the warehouse.

Endpoints

Method	Path	Description
`GET`	`/api/v1/workspaces/{id}/loaders`	List all loaders
`POST`	`/api/v1/workspaces/{id}/loaders`	Create a loader
`GET`	`/api/v1/workspaces/{id}/loaders/{loaderId}`	Get a loader
`PUT`	`/api/v1/workspaces/{id}/loaders/{loaderId}`	Update a loader
`DELETE`	`/api/v1/workspaces/{id}/loaders/{loaderId}`	Delete a loader
`POST`	`/api/v1/workspaces/{id}/loaders/{loaderId}/trigger`	Trigger a sync run
`GET`	`/api/v1/workspaces/{id}/loaders/{loaderId}/runs`	List loader runs
`GET`	`/api/v1/workspaces/{id}/loaders/{loaderId}/runs/{runId}`	Get a loader run
`GET`	`/api/v1/workspaces/{id}/loader-types`	List available loader types

List Loaders

GET /api/v1/workspaces/{id}/loaders

Returns all loaders in the workspace.

Response

[
  {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "workspace_id": "660e8400-e29b-41d4-a716-446655440000",
    "source_id": "770e8400-e29b-41d4-a716-446655440000",
    "name": "Salesforce CRM Loader",
    "loader_type": "salesforce",
    "config": {
      "instance_url": "https://mycompany.salesforce.com"
    },
    "target_schema": "SALESFORCE_RAW",
    "selected_streams": [
      {"name": "Contact", "sync_mode": "incremental"},
      {"name": "Account", "sync_mode": "full_refresh"}
    ],
    "sync_mode": "incremental",
    "schedule": "0 */6 * * *",
    "status": "active",
    "status_error": "",
    "created_by": "880e8400-e29b-41d4-a716-446655440000",
    "created_at": "2024-01-15T09:30:00Z",
    "updated_at": "2024-01-15T09:30:00Z",
    "last_run_at": "2024-01-15T15:00:00Z",
    "last_run_id": "990e8400-e29b-41d4-a716-446655440000"
  }
]

Example

curl -X GET https://your-instance.signalsmith.io/api/v1/workspaces/{id}/loaders \
  -H "Authorization: Bearer <token>" \
  -H "X-Workspace-ID: <workspace-id>"

Create Loader

POST /api/v1/workspaces/{id}/loaders

Creates a new loader to pull data from an external source into a warehouse.

Request Body

Field	Type	Required	Description
`name`	string	Yes	Display name
`loader_type`	string	Yes	Loader connector type
`source_id`	string (UUID)	Yes	Target warehouse source to load data into
`config`	object	Yes	Connection configuration (varies by loader type)
`target_schema`	string	Yes	Schema in the warehouse to write tables to
`selected_streams`	array	Yes	Streams/objects to sync
`sync_mode`	string	Yes	`full_refresh` or `incremental`
`schedule`	string	No	Cron expression for scheduling

Supported Loader Types

Type	Description
`salesforce`	Salesforce CRM objects
`hubspot`	HubSpot CRM objects
`stripe`	Stripe payment data
`shopify`	Shopify e-commerce data
`ga4`	Google Analytics 4
`facebook_ads`	Facebook Ads campaigns and performance
`google_ads`	Google Ads campaigns and performance
`linkedin_ads`	LinkedIn Ads campaigns
`klaviyo`	Klaviyo email marketing
`mailchimp`	Mailchimp email marketing
`postgresql`	PostgreSQL database
`mysql`	MySQL database
`rest_api`	Custom REST API
`google_sheets`	Google Sheets
`s3`	Amazon S3 files

Example

curl -X POST https://your-instance.signalsmith.io/api/v1/workspaces/{id}/loaders \
  -H "Authorization: Bearer <token>" \
  -H "X-Workspace-ID: <workspace-id>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Salesforce CRM Loader",
    "loader_type": "salesforce",
    "source_id": "770e8400-e29b-41d4-a716-446655440000",
    "config": {
      "instance_url": "https://mycompany.salesforce.com"
    },
    "target_schema": "SALESFORCE_RAW",
    "selected_streams": [
      {"name": "Contact", "sync_mode": "incremental"},
      {"name": "Account", "sync_mode": "full_refresh"}
    ],
    "sync_mode": "incremental",
    "schedule": "0 */6 * * *"
  }'

Get Loader

GET /api/v1/workspaces/{id}/loaders/{loaderId}

Returns a single loader by ID.

Update Loader

PUT /api/v1/workspaces/{id}/loaders/{loaderId}

Updates an existing loader configuration.

Delete Loader

DELETE /api/v1/workspaces/{id}/loaders/{loaderId}

Deletes a loader.

Response

{
  "status": "deleted"
}

Trigger Sync Run

POST /api/v1/workspaces/{id}/loaders/{loaderId}/trigger

Manually triggers a loader sync run.

Response

Returns the created loader run object with status 201 Created.

Example

curl -X POST https://your-instance.signalsmith.io/api/v1/workspaces/{id}/loaders/{loaderId}/trigger \
  -H "Authorization: Bearer <token>" \
  -H "X-Workspace-ID: <workspace-id>"

List Loader Runs

GET /api/v1/workspaces/{id}/loaders/{loaderId}/runs

Returns the execution history for a loader.

Response

[
  {
    "id": "aae8400-e29b-41d4-a716-446655440000",
    "loader_id": "550e8400-e29b-41d4-a716-446655440000",
    "workspace_id": "660e8400-e29b-41d4-a716-446655440000",
    "status": "completed",
    "started_at": "2024-01-15T15:00:00Z",
    "completed_at": "2024-01-15T15:05:30Z",
    "rows_loaded": 15420,
    "streams_total": 2,
    "streams_failed": 0,
    "error_message": "",
    "triggered_by": "880e8400-e29b-41d4-a716-446655440000",
    "created_at": "2024-01-15T15:00:00Z"
  }
]

Loader Run Statuses

Status	Description
`pending`	Run is queued
`running`	Run is in progress
`completed`	Run finished successfully
`failed`	Run failed with an error
`cancelled`	Run was cancelled

Get Loader Run

GET /api/v1/workspaces/{id}/loaders/{loaderId}/runs/{runId}

Returns details of a specific loader run.

List Loader Types

GET /api/v1/workspaces/{id}/loader-types

Returns the list of available loader connector types with their configuration schemas and available streams.

Loader Object

Field	Type	Description
`id`	string (UUID)	Unique identifier
`workspace_id`	string (UUID)	Owning workspace
`source_id`	string (UUID)	Target warehouse source
`name`	string	Display name
`loader_type`	string	Connector type
`config`	object	Connection configuration
`target_schema`	string	Warehouse schema to write to
`selected_streams`	array	Streams/objects to sync
`sync_mode`	string	`full_refresh` or `incremental`
`schedule`	string	Cron expression
`status`	string	`draft`, `active`, `paused`, or `error`
`status_error`	string	Error message when status is `error`
`created_by`	string (UUID)	Account that created the loader
`created_at`	string (ISO 8601)	Creation timestamp
`updated_at`	string (ISO 8601)	Last update timestamp
`last_run_at`	string (ISO 8601) or null	Last run time
`last_run_id`	string (UUID) or null	ID of the most recent run

Loader Run Object

Field	Type	Description
`id`	string (UUID)	Unique identifier
`loader_id`	string (UUID)	Parent loader
`workspace_id`	string (UUID)	Owning workspace
`status`	string	`pending`, `running`, `completed`, `failed`, or `cancelled`
`started_at`	string (ISO 8601)	Run start time
`completed_at`	string (ISO 8601) or null	Run completion time
`rows_loaded`	integer	Total rows loaded
`streams_total`	integer	Total streams processed
`streams_failed`	integer	Number of failed streams
`error_message`	string	Error details (if failed)
`triggered_by`	string (UUID) or null	Account that triggered the run
`job_id`	string or null	External job identifier
`created_at`	string (ISO 8601)	Record creation time

Events Govern