The first step to create an A/B test is to think about a metric that is critical to your business’ success. For example, it's a hospital, then “total surgeries” would be an important metric. You may also want to consider testing secondary metrics. For instance, if you are looking at the different types of procedures, the number of case types might be an important metric, perhaps operations by surgeon. Whatever metric you choose, this will serve as your starting point for figuring out what you will test.

Once you’ve chosen an important metric, it’s time to analyzethe funnel where that metric can be measured. Look specifically where customers are dropping off. Gather all of this data, and figure out what area may be good to test. A way to analyze, with out impacting daily business would be via Surgeon Setup Documents or "Preference Cards"as a test on "Costings" will produce results faster. As an example, you can take a commonly used set up card as a template via a word document to start the testing e.g. “BILATERAL ANTERIOR TOTAL HIP REPLACEMENT” if that is the primary way people pick items to be used in the OT and billed against.

Based on data you’ve gathered, come up with a hypothesis for what you want to test, what you think will happen when you test it, and why you think this will happen. Some things you can test include commonly used products or items of value you can measure at present. Testing is a chance to challenge assumptions, so be bold with your hypothesis. Here’s a general template: If we change [this], then [this will happen] because [this reason]. Here are some examples of possible hypotheses: If we change the picked items on this pick sheet from “X” to “Y”, then will this result in a change in cost? A simple yet very important type of test you can set up is called an existence test. In essence, you test whether the existence of a particular element. Here is an example of an existence test hypothesis: If we remove the a specific part or change the quantity , then how does this affect what we do?

Before we can go and test your hypothesis, you need to determine the sample size. It’s important to preface this by saying that A/B testing can be done using analytics. If the tool you are using has these inbuilt it will be easy to run your experiments and gain valuable insights.

Now that you’ve squared away your minimum sample size and your hypothesis, it’s time to set up your test. Set up depends entirely on the A/B testing solution you are using (see the Setting up for A/B Testing section).

No A/B test is complete without a thorough quality assurance process. Without QA, you run the risk of running a faulty test, coming up with faulty results and ultimately coming up with false conclusions that can end up having negative consequences on your business. Run through your test multiple times. Have others test it. Try it on different browsers, devices, IP addresses, etc. Your A/B test platform should have all of the necessary staging requirements to QA your test.

Congratulations! You’ve launched – and now it’s time to monitor your test, but don’t try to call the test too early. Even if you reach your minimum sample size, there are a load of factors that may be affecting performance, and the goal is to ride out the seasonality of those effects. In general, 7 days is the minimum amount of days the test should be live, and ideally for two business cycles to capture normal fluctuations.

Before you can call a test, it’s important to use an A/B testing statistical significance calculator. As a rule, a test that shows a gain with 95% certainty can be declared a winner, although you can go even lower based on your risk tolerance. As noted above, be sure to run your test long enough before checking the statistical significant of the results. Regardless of your results, you should always take time to analyze the results. Ask yourself: What did we learn about our users? What can we do to improve our processes? How will these insights inform future testing? For example, if this was your hypothesis: If we move our customer validation logos above the fold on our marketing asset, then more people will convert because they will see the logos first and therefore trust us more.

Finally, it’s important to thoroughly document your tests, conclusions and analysis, in order to build a testing culture in your company. Testing early and often will help you keep up with changes in the market and your customers’ needs and attitudes, and can help drive your organization’s growth. With this in mind, make it a goal to always be testing some aspect of your business. You may be surprised by the insights you uncover.