TidepoolIssue 14: The case for slower reps
A/B tests your AI runs for you
Subject lines, senders, designs, even whole sequences. Test them on a sample, and the winner goes to everyone else.
Run one now
Test three subject lines for this month’s newsletter on 20% of active members, by clicks, and send the winner to the rest.
- Testing
- Subject line
- Audience
- Active members · 8,412 people
- Sample
- 20% first, the winner to the other 80%
- Winner by
- Human clicks, at 95% confidence
- AIssue 14: The case for slower reps 4.1%
- BWhy slower reps make you strongerWinner 6.3%
- CSlow down. Get stronger. 5.0%
54% more clicks than version A
- Testing
- Sender name
- Audience
- Bought in the last year · 4,216 people
- Sample
- 20% first, the winner to the other 80%
- Winner by
- Human opens, at 95% confidence
- AKiln Studio 31.0%
- BRita at Kiln StudioWinner 38.2%
23% more opens from a person’s name
- Testing
- Design
- Audience
- All members · 12,960 people
- Sample
- 20% first, the winner to the other 80%
- Winner by
- Human clicks, at 95% confidence
- AProduct spotlight 9.8%
- BPhoto announcementWinner 12.7%
30% more clicks for the photo
- Testing
- Journey paths
- Audience
- Everyone who signs up
- Split
- 45% · 45% · 10% control
- Winner by
- They log a workout, at 95% confidence
- AThree emails over a weekWinner 47.0%
- BOne welcome email 41.0%
- CControl: no emails 22.0%
47% reached the goal on path A, now everyone gets it
TidepoolWhy slower reps make you stronger
TidepoolSlow down. Get stronger.
Kiln Studio20% off the whole shop
Rita at Kiln Studio20% off the whole shop
TidepoolNew in Tidepool: Streak freezes
TidepoolStreak freezes are here
TidepoolYou’re 2 steps from finishing week one
TidepoolWelcome to Tidepool. Start here.
Real readers only
Bots and privacy proxies don’t count.
Guardrails
A version that drives unsubscribes can’t win.
Whole sequences too
Split a Journey and measure against its goal.
Questions about email a/b testing
What is email A/B testing?
Sending two or more versions of an email to different people to see which works better, then sending the better one to everyone else.
What can I test?
In a Broadcast: the subject, design, content or sender name, with 2 to 4 versions. In a Journey: whole paths of emails, plus a control group that gets no email, to see whether emailing helps at all.
How does Banger pick the winner?
You choose the metric: human clicks, human opens or replies. Banger picks once one version leads with 95% confidence, or at the deadline you set (4 hours by default). You can also pick it yourself.
How big should a test be?
Aim for at least 100 people per version. Smaller samples give wide ranges, so the result can swing.
Can my AI set up a test?
Yes. Ask it to test two subject lines on 20% of the audience and pick the winner automatically. It sets the sample, the metric and the deadline for you.
Are A/B tests included in the free plan?
Yes. Every feature is on every plan. Free includes 500 emails a month, up to 100 a day. Premium is $20 a month billed yearly, or $24 month to month, and includes 50,000 emails a month.