• test
  • BEST PRACTICES
  • INTEGRATIONS
  • DIGIOH
  • FACEBOOK
  • GIGYA
  • GOOGLE ADWORDS
  • GOOGLE TAG MANAGER
  • INKIT
  • INSTAGRAM
  • JUSTUNO
  • KICKDYNAMIC
  • MAGENTO
  • MPARTICLE
  • PERSADO
  • SEGMENT
  • SHOPIFY
  • SMS INTEGRATIONS
  • WORDPRESS
  • ZENDESK
  • MOBILE
  • MESSAGES
  • ANALYTICS
  • AUDIENCES
  • SETTINGS AND REFERENCE
  • RELEASE NOTES
    Release Notes
  • Recently UpdatedDocumentation
  • FOR DEVELOPERS
    FOR DEVELOPERS OVERVIEW
  • API BASICS
    Introduction to the Sailthru API API Technical Details API Postbacks User Queries for Job and List API Error and Response Codes API Test Page
  • API ENDPOINTS
    ad/plan blast blast_repeat content event include job list preview purchase return send settings SPM stats template trigger user
  • API CLIENT LIBRARIES
    API Client Libraries Overview Java Library JavaScript API Library .NET Library Node.js Library Perl Library PHP5 Library Python Library Ruby Library
  • ZEPHYR GUIDE
    Zephyr Overview Common Zephyr Examples Zephyr Examples Library Zephyr Template Variables Zephyr Expressions And Operators Zephyr Control Structures Zephyr Assignment Operator For Variables Single Vs. Double Braces “Profile” Object “Message” Object “Blast” Object “Feed” Object Link Rewriting With Zephyr Set User Vars or List Membership On-Click with URL Parameters Forward To A Friend
  • ZEPHYR FUNCTIONS
    Zephyr Functions Index abs ad adinfo api_event api_send api_user append_user_var assert base64_decode base64_encode bucket_list cancel compare contains content_intersect date dedupe distance exp filter filter_content first h horizon_count horizon_interest horizon_select horizon_set_interest int intersect join keys length list Function lower load_content map md5 message_id number personalize promotion public_share purchased_items push random range replace round set sha1 sha256 shuffle signup_confirm slice social_share sort split sqrt strip_tags strpos strrpos substr text time title type u upper user_engagement user_geo_home user_geo_select_region values
  • ZEPHYR TUTORIAL
    Zephyr Tutorials Lesson 1: Zephyr Basics Lesson 2: Sailthru Data Basics
  • Home
    Search Results
  • Web
    Sailthru On Your Site – Overview
  • SITE PERSONALIZATION MANAGER
    Site Personalization Manager Overview Site Personalization Manager Implementation Create a Rule Configure a Section View Site Personalization Analytics Overlays
  • PERSONALIZATION ENGINE
    Personalization Engine Overview Personalization Engine Algorithms Personalization Engine Setup Personalization Engine Meta Tags Personalization Engine Meta Tag Examples Personalization Engine Auto Tagging Personalization Engine Setup: Include/Exclude Pages Personalization Engine in Email Templates Custom Personalization Engine Tags Personalization Engine Content Library Sailthru.track (for Legacy Horizon JS) Personalization Engine via the API
  • ANONYMOUS USER CONVERSION
    Anonymous User Conversion Overview Adding Anonymous Data To A New User Adding Anonymous Data To An Existing User
  • SCOUT
    Scout Onsite Module
  • CONCIERGE
    Concierge Onsite Slider Concierge Onsite Signup
  • Email
    Email Overview
  • TEMPLATES
    Templates Overview Build a Template Email Composer, the Drag-and-Drop Template Builder Optimize Your Templates for Mobile Social Media Sharing Track Opens with a Beacon Image Insert Reusable Code or Content with Includes Track Link Clicks in Sailthru and Google Analytics Organize Templates Using Labels
  • CAMPAIGNS
    Campaigns Overview Regular Campaigns Recurring Campaigns AB Split Campaigns AB Winner Campaigns AB Auto-Winner Campaigns View and Organize Campaigns
  • TRIGGERED MESSAGES
    Triggered Messages Overview Transactional Messages Overview Set Up a Triggered Message Creating Template Triggers for Messages Abandoned Shopping Carts A/B Split Testing Triggered Emails Smart Strategies Force Send Purchase Emails
  • LIFECYCLE OPTIMIZER
    Using Lifecycle Optimizer Lifecycle Optimizer Step Glossary Automate Abandoned Cart Reminders Automate Browse-Abandonment Responses
  • PROMOTIONS
    Promotion Codes
  • AD TARGETER
    Ad Targeter Overview Create an Ad Plan Create an Ad Plan Schedule Target Users with Ad Targeter
  • DATA FEEDS
    Data Feeds Overview Set Up a Data Feed Create a Content Feed from Your Content Library Use Feeds and Other Content Data in Templates Build a Campaign from an Edited Feed using Recommendation Manager Call An External Data Feed Example Feeds
  • DELIVERABILITY
    Deliverability Overview Improving Deliverability Configure DNS Records
  • Audience
    AUDIENCE OVERVIEW
  • MANAGING USERS
    Managing Users Overview User Profile Lookup and Data Glossary Adding Users to Sailthru and Sailthru Lists Set Custom Fields (Variables) on Users Set Custom Signup Dates User Engagement Levels Add Users Using a Signup Form Allowing Users to Opt Back In User Opt-Out Levels
  • MANAGING LISTS
    Managing Lists Overview Natural Lists Smart Lists Primary and Secondary Lists: What’s the Difference? Snapshot Reports Merge Multiple Lists
  • USER DATA EXPORTS
    Export Lists and User Profile Data List Export Variable Glossary
  • AUDIENCE BUILDER
    Using Audience Builder Audience Builder Filter Reference What’s New in the New Audience Builder
  • PREDICTION MANAGER
    Prediction Manager Overview User Predictions The Prediction Manager Interface AB Testing with User Predictions Product Predictions
  • Analytics
  • ANALYTICS & REPORTING
    Analytics Overview Analytics Glossary Realtime Dashboard Weekly Performance Report
  • USAGE DASHBOARD
    Send Volume Usage
  • USER REPORTS
    User Reports Overview User Lookup Lists Report List Growth Report Source Signup Report Snapshots List List Performance Report Jobs List Postback Log
  • CAMPAIGN REPORTS
    Campaign Reports
  • TRANSACTIONAL REPORTS
    Transactional Email Reports
  • PURCHASE REPORTS
    Use the Purchase Log to View/Export Purchase Data
  • SAILTHRU SYSTEM
    System Status Dashboard and Alerts
  • DATA EXPORTER
    Sailthru Data Exporter
  • My Account
    My Account Overview
  • HOSTED PAGES
    Sailthru Hosted Pages Overview Build a Signup Page Build a User Management Page for Subscription Preferences and Opt-Outs Build a Thank You/Confirmation Page Data Validation for HTML Set User Variables from Hosted Pages Domains and URLs for Hosted Pages
  • ACCOUNT MANAGEMENT
    Account Management Overview Add and Manage User Accounts and Permissions Manage List and Template Access with Teams Account Settings Create and Manage Support Cases Universal Links and Your iOS App
  • A/B Split Testing Transactional EmailsAB Split TestsAB Winner and AB Auto Winner Campaign TestsAudience Builder OverviewUsing Audience BuilderAudience Builder Criteria GlossaryBulk Updating in Audience BuilderGeolocation Region Codes and SyntaxCampaign Editor OverviewFilter and Label CampaignsCampaign Editor InterfaceChanging A Campaign’s NameCampaign Type InterfaceSelecting A Channel And Campaign TypeConfigure InterfaceBuilding A Campaign From A TemplateBuilding A Campaign From ScratchSplit Your List for AB TestsEditing the BasicsEditing the Personalize Sub-SectionConfirmation InterfaceSchedule Your Campaign on the Confirmation PageUnscheduling A Scheduled CampaignExploring A Sent Campaign’s MetricsDesign InterfaceFilling The HTML Sub-SectionFilling The Text Sub-SectionFilling The Content Sub-SectionFilling The Advanced Sub-SectionUsing The Editor Mode’s OptionsUsing The Image LibraryUsing The Validate Sub-SectionUsing The Test Vars Sub-SectionSearching Editor ModeChanging User Perspective In Preview ModeTest Sends for CampaignsSchedule InterfaceSetting A Campaign’s Schedule TimeUsing Personalized Send TimeCarnival SettingsPausing Push NotificationsThrottling Push NotificationsMobile GlossaryUsing Carnival with SailthruSelecting Your AudienceCampaign Editor Overview
  • GETTING STARTED
  • CAMPAIGN TYPE SECTION
  • AUDIENCE SECTION
  • CONFIGURE SECTION
  • A/B TESTS
  • DESIGN SECTION
  • SCHEDULE SECTION
  • CONFIRMATION SECTION
  • Transactionals OverviewSet Up a Transactional MessageGETTING STARTED

    A/B Split Testing Transactional Emails

    Continuously improve the design and creative of your transactional emails by A/B testing different versions. What subject line is most inviting? What’s the best place to place a link to get an invited user to join your site?

    Best Practice: Use Lifecycle Optimizer to test triggered and transactional messages and message series, such as welcome and post-purchase series.

    Read on to learn about our legacy method of API-based A/B Split testing with transactional/triggered messaging.

    The basic idea of A/B testing is simple. Create two slightly different versions of a transactional template and set up to send using the API. Reporting will determine which one gets a better response from your users, usually measured by click and open rates.

    For more information on mass mail campaigns, see A/B Split Testing Campaigns.

    Setting Up the A/B Test

    1Create a new template or edit an existing one in the Templates tab.

    From inside the Template Editor, click the A/B Test button and hit OK at the confirmation prompt.

    27_02_05 A:B Test Transactonal-A:B Test Button

    2You will confirm via a dialog box to create two versions of your template.

    The names of each template will be appended with A and B. You’ll be inside the template editor of version A. To view both, save and click out to the Templates tab in the main menu.

    28_02_05 A:B Test Transactional-Segments listed as templates

    Using the Rename button to change the name of a template to end with [A] or [B] does not make it an a/b cell; the A/B Test button must be used.

    3Edit one version to be slightly different than the other.

    You can give it a unique subject line as well as change the message content.

    A/B Testing in Action

    For a given transactional email sent via the API, Sailthru will pick randomly between the different A/B versions. After a period of time, you will be able to compare the results and determine which is the most effective template.

    Note: A user’s placement in the A or B grouping will persist for all future transactional test sends.

    Reporting on Testing

    In the Reports tab, click on Transactional A/B Testing Report to see analytics on the different versions.

    End Testing

    Once you’re satisfied that you know which is the better performing template and you want to stop testing, reach out to your Customer Success Manager and they will mark the template you no longer want to send as inactive. Whichever version is left will be sent from this point forward.

    Note: If you mark both versions as inactive, Sailthru will send neither. Do not delete the template as you will lose its historical performance.

    Top