Splits

Splits let you divide an audience into random, percentage-based groups for A/B testing, holdout experiments, and controlled rollouts. Each split creates two or more non-overlapping subsets of the audience that together cover the full membership.

What Is a Split?

A split takes an existing audience and partitions its members into groups based on percentages you define. The assignment is random but deterministic — the same entity always lands in the same group as long as the split configuration doesn’t change.

┌─────────────────────────────────┐
│      Original Audience          │
│         10,000 members          │
└─────────┬───────────────────────┘
          │
    ┌─────┴─────┐
    ▼           ▼
┌────────┐  ┌────────┐
│ Group A │  │ Group B │
│  80%   │  │  20%   │
│ 8,000  │  │ 2,000  │
└────────┘  └────────┘

Each group acts as its own audience and can be synced independently to different destinations or different treatments.

Use Cases

A/B Testing Campaigns

Test two versions of a marketing campaign by splitting the audience into control and treatment groups:

• Group A (50%) — Receives Campaign Version A
• Group B (50%) — Receives Campaign Version B

Compare conversion rates between the groups to determine which version performs better.

Holdout Experiments

Measure the incremental impact of a campaign by withholding it from a percentage of the audience:

• Treatment (90%) — Receives the campaign
• Holdout (10%) — Does not receive the campaign

Compare outcomes between the treatment and holdout groups to measure true campaign lift.

Controlled Rollouts

Gradually roll out a new campaign or treatment to increasing percentages of the audience:

• Week 1: 10% receive the new campaign
• Week 2: 25% receive the new campaign
• Week 3: 50% receive the new campaign
• Week 4: 100% receive the new campaign

If issues arise, the rollout can be paused before reaching the full audience.

Multi-Variant Testing

Test more than two variants simultaneously:

• Control (25%) — No campaign
• Variant A (25%) — Email campaign
• Variant B (25%) — Push notification
• Variant C (25%) — SMS campaign

Creating a Split

Step 1: Select the Audience

Navigate to Segment > Splits and click Create Split. Select the audience to split.

Step 2: Define Groups

Add groups and assign percentages:

Requirements:

• At least 2 groups must be defined
• Percentages must sum to exactly 100%
• Each group must have a minimum of 1%
• Each group needs a name (used as the group identifier in syncs and reporting)

Step 3: Configure Randomization

Choose the randomization strategy:

For most use cases, hash-based is recommended because it provides stable group assignment — a customer in Group A today will still be in Group A tomorrow, ensuring consistent treatment.

Step 4: Save

Click Save to create the split. SignalSmith evaluates the split immediately, assigning each audience member to a group.

How Randomization Works

Hash-Based Assignment

SignalSmith computes a hash of each entity key and uses the hash value to assign the entity to a group:

1. Compute hash = SHA256(entity_key + split_id)
2. Map the hash to a value between 0 and 100
3. Assign to a group based on the cumulative percentage ranges

For example, with Group A at 80% and Group B at 20%:

• Hash values 0-79 → Group A
• Hash values 80-99 → Group B

This approach ensures:

• Determinism — Same entity key always produces the same hash, so group assignment is stable
• Uniformity — SHA256 produces a uniform distribution, so groups are accurately sized
• Independence — Different splits (different split_id) produce different assignments for the same entity

Re-Splitting on Membership Changes

When the underlying audience membership changes (new members join, existing members leave):

• New members are assigned to a group using the same hash-based logic
• Departed members are removed from their group
• Existing members stay in the same group (hash is deterministic)

This means group proportions may drift slightly as the audience membership changes. If exact proportions are critical, you can re-evaluate the split to rebalance.

Syncing Split Groups

Each split group can be synced independently as if it were its own audience. When creating an audience sync, you can select a split group as the source instead of the full audience.

This lets you:

• Send Group A to one destination (e.g., treatment campaign) and Group B to another (e.g., no campaign)
• Send different creative variations to different groups
• Apply different sync modes to different groups

Monitoring Splits

The split detail page shows:

Editing and Deleting Splits

• Editing group percentages — Rebalances all group assignments. Existing syncs continue with the new groups.
• Adding a group — Splits members from existing groups to populate the new group.
• Removing a group — Redistributes members from the removed group into remaining groups.
• Deleting a split — Removes the split and all group assignments. Active syncs targeting split groups must be removed first.

Best Practices

• Use hash-based randomization for experiments — it ensures stable group assignment across evaluations
• Keep holdout groups small but statistically significant — a 10-20% holdout is usually sufficient for measuring lift
• Don’t change split percentages mid-experiment — this invalidates the statistical comparison between groups
• Document your experiment hypothesis in the split description so team members understand the purpose
• Allow sufficient time for experiments to reach statistical significance before drawing conclusions

Next Steps

• Audiences — Build the audiences to split
• Audience Syncs — Sync split groups to destinations
• Priority Lists — Manage overlap between audiences