What is A/B Testing? Why is it Important?
A/B testing is a method of assessing multiple variables in your job posts (such as images, post titles, posting time, etc.). In this method, a specified variable is compared between two similar ads, to help identify which version of the variable improves target metrics. The rest of the content remains largely the same for quality control.
Running A/B tests in your recruiting initiatives is a great way to learn what content will drive more traffic to your posts and generate more applicants. Even small tweaks to a post title, image, or call-to-action can significantly affect the number of applicants your company attracts – and this data can provide a substantial competitive advantage for your company. Aside from finding what works best for your company, it’s likely that your competitors aren’t regularly A/B testing, which gives you even more of an edge.
A/B Testing Ground Rules
What variables should you test? How long should those tests last? How do you know if your findings are statistically significant? A/B testing can seem confusing or daunting at first, but don’t worry – we have some guidelines for you to consider before implementing:
1. Only test one variable at a time. Let’s say you have a new job opening you’d like to post. You might decide to assess the day of the week that you’re posting on, and you may also be interested in testing whether or not including an image in the post improves conversions. Both of these things may be essential elements to test, but if you conducted both tests simultaneously, you’d have invalidated the results. How would you know which change ultimately impacted the conversion rates? Maybe it was the day of the week? Perhaps it was the image? Or perhaps even both? For this reason, you’ll want to test only one hypothesis at a time, ensuring that your results are more accurate and that you may draw reliable conclusions.
2. Conduct only one test at a time. This idea is similar to the point above, but with a broader scope. Let’s use the example above, and say that you decided to test whether including an image makes a difference in your conversions, and you’ll save the day-of-the-week experiment for another time. You’re off to a great start, testing one post element at a time! Now, it’s time to think further down the funnel and make sure that your internal recruiting processes are not also changing or being tested. Since your interactions with applicants continue after RocketPost, you’ll want to be sure that you view this process holistically and see where other parts of your recruitment process may unintentionally affect the results of your test.
3. You can A/B test entire posts. You may find insightful results by testing single post elements like the post title or some specific call to action, but you may also want to consider making your entire post one variable. Instead of testing single design elements, you might try designing two completely different posts and testing them against each other. Every company is different, so depending on the information that’s most important to you or the questions you’d most like to answer, testing whole posts against each other may be more meaningful to you than testing individual post elements.
4. Split your sample group randomly. To achieve conclusive results, you need to test with two or more audiences that are equal. For example, let’s say you’re A/B testing the time of day that yields the most post impressions – you want to see if a post made at 9 AM gets drowned out by posts made later in the day, say at 4 PM. Both variations must have as similar an audience as possible. That means you’ll want to test in the same market if possible.
5. Test simultaneously. Timing plays a significant role in your test results – things change week-to-week, month-to-month, and so on. If you were to run test A during one month and test B a month later, you wouldn’t know whether the results were due to the variable you were testing, or because applicant behaviors changed since the last month. A/B testing requires you to run the two variations as close to the same time as possible to trust your results. If you’d like assistance setting up an A/B test or analyzing your results, the RocketPost Success team is here to help!
What Elements of My Posts Should I Test?
As you create job posts, you’ve probably wondered about the elements you can optimize to increase conversion rates. Should you change the title of the post to draw in more clicks? How can you modify the language on the call-to-action to get more applicants? What if you removed all images from the post? Every company is different and has different needs. Focus on testing elements that could have the most impact on your result (impressions, conversions, whatever matters most to you). Still stuck? Try a few of these:
- Post Titles
- Images
- Craigslist Category
- Employment Type
- Time of Day to Post
- Day of the Week to Post
- Benefit & Compensation Statements
- (CTA) Call-To-Action
Running and Evaluating Your Tests
Now that you know why you should run A/B tests, best practices for running tests, and what variables you can optimize, let’s look at how you do this in RocketPost. How do you create multiple versions of the same element you are testing? How do you find results? Below, we’ll go over a few examples. Ultimately, you will want to work with the RocketPost Success Team to discuss your test and measure your results.
Example: Conducting an Image test on RocketPost
Below, you’ll see screenshots of two different Templates in RocketPost. Notice that all post elements are the same except for the image in the post. Each of these Templates will be uniquely named so that we can measure which image improves post-performance most. RocketPost makes it easy to test post elements against each other using our Template feature.
Conclusion
A/B testing allows you to collect insightful data, solve problems, and challenge assumptions. The greatest gift that the practice of A/B testing gives you is the ability to optimize processes that matter. Together we have covered the basics of A/B testing, and now you probably have many things you want to test.
We know this is a lot of information, but the results of your A/B tests can have a significant impact. The findings from these can play a big part in optimizing ad performance, both in the short-term and long-term.
If you’re looking to set up testing, analyze results, or just want further clarification on anything mentioned above, we’re here to assist you. Now, all that’s left to do is talk to your RocketPost Success Team!