Grace Cooper:
If you’re responsible for outreach in your office, chances are you’ve heard of A/B testing but may not have had the time to research or test it out. Well don’t worry, we’ve written out everything you need to know, from what it is to best practices, down below.
What is A/B testing?
A/B testing is when you send two versions of the same outreach, version A and B, to a small portion of your distribution list to test which one is more successful, and then send the better version to the remaining portion of the list. Version A and B can differ in content or design changes but are ultimately the same email. The “success” of each version can be determined by a variety of factors, such as most viewers, most click throughs, or most survey takers.
How does it work in IQ?
When done manually, A/B testing can be a time-consuming process. Luckily, IQ does a lot of the work for you through automation. With IQ, users simply have to:
- Create their two versions
- Select the sample size, the length of the test, and the criteria for determining the winning outreach (all accessible from one pop-up window
- Build and send
Why should you use it?
A/B testing is best used for testing small changes between two mostly similar emails. Are you ever creating an outreach and can’t decide which word would be stronger in your call to action or whether a green button is better than a red? Well A/B testing gives you the opportunity to try both and get the answer.
A/B Testing Best Practices
As you may have gathered already, A/B testing is conducted much like a scientific experiment. Therefore, the IQ user must behave like a scientist, preparing for the experiment and analyzing the results. Here are few tips to getting the most out of your A/B test.
- Test only one variable. Just like a scientific experiment, it’s best to change just one element between your A and B version, whether it’s the subject line, header image, etc. That way you can attribute the success of one version over the other to that one change, rather than having to guess.
- Only test when there’s a question. A/B testing is only useful when you have a defined reason for doing it because random reasons will only give you random results. For instance, don’t use two different images in your A and B version just because you like the way they look. Use them because they have significant differences that might incite significantly different responses, such as one is an image from your district but the other has an image with people. Both will be relatable to the reader but which one will resonate more? That’s where the A/B test comes in.
- Be creative. As a caveat to the previous tip, don’t be afraid to be creative in your outreach testing, whether its with design or content. Whether your struggle with low open rates or maintaining steady engagement, there’s always room for improvements when it comes to engaging with your constituents.
- Predict your results. It may seem unnecessary, but it’s important to formulate a hypothesis before conducting an A/B test. As you know, testing to test won’t get you anywhere–predicting what will happen and why forces you to draw on your knowledge of your audience and your office’s general outreach strategy. If you’re right, you can take that success and use it as a reason to test something new and different. If you’re wrong, take the time to evaluate and improve your outreach strategy going forward.
- Analyze your results. Like anything you do in IQ, the results of your A/B test are kept for later comparison and analysis.
- Strategize and repeat. Use the results of your A/B test to inform your next outreach and test another component of newsletter or press release. Ultimately, one A/B test is not going to give you profound insight into the audience you’re reaching, but combining the results of several A/B tests over a period of time will help you to better know your audience and apply that knowledge to your outreach strategy.
By now you should have a good idea of what A/B testing is, why it’s important, and where to go from here. Keep in mind that A/B testing, like other outreach tools, is not a fix-all solution but rather an easy and efficient way to supplement your outreach strategy. Want to learn more about A/B testing in IQ? Contact your ITC or the help desk for more information.
