A/B test your messages
Split a step into Variant A and Variant B. Hollerly sends each to half your leads and picks the winner by reply rate after 30 sends each. Here's how.
A/B tests are on Pro and Agency.
Set up a test
- Open the campaign editor and click Edit message on a step.
- Turn on A/B test.
- Write or direct Variant A and Variant B. Half your leads get each one.
Each lead always gets the same variant, so follow-ups stay consistent.
How the winner is picked
- Both variants need at least 30 sends before Hollerly calls it.
- It compares reply rates with a standard statistical test at about 90% confidence.
- Once a winner is clear, Hollerly switches everyone to it automatically on active campaigns. It checks once a day.
Watch and decide yourself
The A/B tests section on the campaign page shows each variant's sends, replies and reply rate, with a badge such as "variant A won", "B is ahead" or "collecting data". You can decide early with Use A for everyone or Use B for everyone, or click Keep testing.
Tips
- Test one thing at a time, such as the opening line or the call to action.
- Small campaigns may never reach 30 sends per variant. Test on a list of at least 100 leads.
If you move to Starter
Starter doesn't include A/B tests. A campaign with a test in it stops sending after the change: its leads wait where they are, and drafting new messages asks you to upgrade. Turn the test off in the campaign editor, or upgrade, to carry on. The same applies to conditions and personalized images.