Setting up an A/B test on Spark

This article will show you how to create a new A/B test in Spark and will cover the various options that you can use to test your work based on your marketing goals.

Before you get started, Make sure you have created a page that you want to test.

Step one: creating the test

Go into the page you've created. Hover over the section that you want to test. In the toolbar, click the beaker icon. You should instantly see a new toolbar drop down that says "control" and "variant".

  • The control: this is your original content.
  • The variant: this is your hypothesis that you're testing

Change the variant content to add your test material in. Once complete, click on the settings option. A sidebar should pop out on the right hand side.

Step Two: Choosing a Goal

There are three goals that you can choose from when running a test:

  1. Fire a conversion if the user reaches a certain page
  2. Fire a conversion if the user fills out a specific form
  3. Fire a conversion if the user clicks on a button

If you're not sure which one to pick, read this article about picking the right strategy for your A/B test.

Step Three: Throttling

Once you've selected a goal, choose the amount of traffic you want to route to the control and variant. In most cases we recommend a 50/50 split and have set that as the default. However, if you have significant web traffic you may want to route a smaller percent to your variant. It's up to you.

Steve Four: Name and Launch

The last step is to create a name for your test and publish it. We recommend picking a descriptive name, which becomes helpful as you run numerous concurrent tests across your marketing site, landing pages, and blog content

Important note about A/B testing:

Once you publish the test you will not be able to make further edits to the section until the test reaches a winner or you end the test and manually select a winner. You can make edits to other sections, though we do not recommend it as it could impact the statistical integrity of the test.