A/B Split Testing Transactional Emails

Welcome to our updated Transactionals pages! Please take our brief survey at the bottom of this page.

Continuously improve the design and creative of your transactional emails by A/B testing different versions. What subject line is most inviting? What’s the best place to place a link to get an invited user to join your site?

The basic idea of A/B testing is simple. Create two slightly different versions of a transactional template and set up to send using the API. Reporting will determine which one gets a better response from your users, usually measured by click and open rates.

For more information on mass mail campaigns, see A/B Split Testing Campaigns.

Setting Up the A/B Test

1Create a new template or edit an existing one in the Templates tab.

From inside the Template Editor, click the A/B Test button and hit OK at the confirmation prompt.

27_02_05 A:B Test Transactonal-A:B Test Button

2You will confirm via a dialog box to create two versions of your template.

The names of each template will be appended with A and B. You’ll be inside the template editor of version A. To view both, save and click out to the Templates tab in the main menu.

28_02_05 A:B Test Transactional-Segments listed as templates

Using the Rename button to change the name of a template to end with [A] or [B] does not make it an a/b cell; the A/B Test button must be used.

3Edit one version to be slightly different than the other.

You can give it a unique subject line as well as change the message content.

A/B Testing in Action

Each time a transactional email is scheduled via the API, Sailthru will pick randomly between the different A/B versions. After a period of time, you will be able to compare the results and determine which is the most effective template.

Reporting on Testing

In the Reports tab, click on Transactional A/B Testing Report to see analytics on the different versions.

End Testing

Once you’re satisfied that you know which is the better performing template and you want to stop testing, reach out to your Customer Success Manager and they will mark the template you no longer want to send as inactive. Whichever version is left will be sent from this point forward.

Note: If you mark both versions as inactive, Sailthru will send neither. Do not delete the template as you will lose it’s historical performance.

Thank you for using our updated Transactionals pages. Please take our brief survey below.
Top