A/B Testing

Basics

A/B testing helps you optimize your website through experimentation. Imagine if you weren’t sure whether a signup button should be red or green. With A/B testing you would test both versions simultaneously and then look at the data to determine which button color yielded better results. This is usually more efficient and more accurate than having a debate amongst the design team about which button color might be better. Although we will continue to use the simple red button vs green button example throughout this documentation A/B tests can test much more sophisticated behavior and can have more than two variants.

There are a few basic steps involved in A/B testing:

  1. Assigning each visitor to a variant (red button or green button)
  2. Making sure that the visitor always sees that same variant. Ideally you don’t want someone to see a red button during one visit and then a green one the next time.
  3. Displaying the appropriate variant.
  4. Recording which variant which users saw.
  5. Recording the target metrics that we’re interested in. In this case we might want to see which button color yields more signups, so we need to also record signups somehow.
  6. Viewing the results and making a decision. After you run the experiment and collect enough data, you’ll need to view the data and make a final decision on whether you want to go with a red button or a green button.

KISSmetrics will take care of nearly every single one of those steps except displaying the actual variant (you’ll need to do that)

Filed under Advanced

Search

Learn more about KISSmetrics

Get started now