BUILDER

Resources

×

Visual CMS

Drag-and-drop visual editor and headless CMS for any tech stack

Theme Studio for Shopify

Build and optimize your Shopify-hosted storefront, no coding required

Resources

Blog

Get StartedLogin

☰

Home

Forum

Blog

Github

Login

Signup

This page was made in Builder!

A/B Testing

This guide will walk you through how to leverage A/B testing in Builder. A/B testing is a as a data-driven approach to testing and measuring different content and experiences to help you make smart, impactful improvements to your site or app. 

Creating variations

In order to leverage A/B testing, you first want to create different variations of a page or piece of content in order to measure metrics between the two.

In the Builder editor, click the A/B Tests tab and select Add A/B Test Variation.

You can rename your new variation (titled Variation 1 by default), adjust the test ratio. You can click Add A/B Test Variation to create more variants.

Note: The default variation is the "control" meaning it will be exposed to search engines, users with javascript off, and/or tracking disabled

Navigating back to the Edit tab, you can toggle between the two variations and make adjustments. When ready, click Publish to start the test or you can schedule it to publish later You can learn more about scheduling here.

A/B Test Metrics

To determine how your test is performing, click on the A/B Tests tab, and then click the "View Results" button underneath the sliders. This will take you to the insights tab, where you can see results for the test. At first you will not see many visitors or much of anything there, because the test has not been running long enough. But after a while, you will see some statistics on how your tests are doing. It will look like the sample data below.

Note that for conversion metrics on custom tech stacks you'll need to integrate conversion tracking. For Shopify hosted stores, this integration is done for you automatically

We can use this data to determine which variation is performing better by comparing the conversion stats between the variations. Depending on what is important to you and your store, you might choose to focus on conversion rate, number of conversions, or total conversion value. Or maybe you are just interested in the number of clicks a particular version got. No matter what metric you choose to focus on, make sure you give the test a long enough time to run and get enough visitors.

Once you have determined the winning variation, it's time to roll out the winner to all your visitors. If you want only show the winning variation to new visitors, or want to keep all the variations around to possibly use in the future, feel free to adjust the sliders in the A/B test tab to the desired ratios. If you would rather remove all the variations except for the winning one, click "Choose Winning Variation" on the A/B test tab. This will open a modal that allows you to select a winning variation and remove all others, ending the test.

Ending an A/B test

In order to end your A/B test and still maintain the data, duplicate your page by clicking the three dots in the top right corner.

👉Tip: It's important to create a duplicate page before ending your a/b test in order to maintain and reference the metrics later

On the duplicated page, click "Choose Winning Variation" on the A/B test tab. This will open a modal that allows you to select a winning variation and remove all others, ending the test.

Why is our solution different?

  • 0 performance impact. Unlike any other a/b testing tool, our ab tests do not block and slow down your site
  • The Builder editor serves as a single source of truth between its easy drag and drop interface and metrics page
  • SEO friendly!
  • No DOM blocking manipulation or flashing of unwanted content ever

Up next

Targeting and Scheduling
Was this article helpful?