What is A/B Testing?
“Test one change at a time or try Predictive Analytics”
A/B Testing is the contemporary name for the classic scientific 'controlled experiment'. Classic A/B Testing or Split Testing was publicly used in marketing in the 1960’s & 70’s to promote “washing powders”.
The classic A/B Test was to prove the common claim:
"We took two ordinary white shirts and washed one using detergent A and the other in detergent B. See how much whiter shirt B is".
In the online world, A/B Testing has come back into vogue by comparing two versions of a web page or a banner to see which one performs better.
In an A/B test, visitors get assigned to either web page “A”, which is usually the control group, or web page “B” which has the variable being tested (the altered content). A/B Testing, or Split Testing, is a way to confirm that an outcome was caused by one variable, provided care is taken to remove systematic bias.
More about A/B Testing
A/B Testing is a very simple experiment to determine which out of two options is the best. The main disadvantage of A/B Testing or Split Testing is that you can only change one variable at a time.
Using the washing powder example, the experiment fails to include different factors like water temperature, types of fabric, etc. So, it may be the case that there is only one circumstance (the one shown on the ad) where washing powder B actually makes shirts 'whiter' than washing powder A.
In the same way that it was a powerful advertising tool used in the 1960’s and 1970’s, A/B Testing is being promoted again for many online experiments.
A/B Testing and Predictive Analytics complement each other.
A combination of both research methods is the most powerful elixir to optimize Click-Through Rates.
A/B Testing is virtually useless when it comes to testing a range of new Adword Ads, for example. It is much too inefficient, due to the nature of A/B Testing.
If you were to use A/B Testing on a new AdWords campaign, you would only ever to be able to change one variable – one word – at a time. An example would be adding Affordable in the variation Headline.
Testing one option at a time is impractical and inefficient. The combinations that you can have for different Call to Actions, spacing methodologies, and other functional aspects of an ad are incredibly numerous. Especially when you include the variations that can happen between the three (3) crucial aspects of any ad (Headline, URL, and Description), you will discover that there are literally thousands of combinations.
With a Predikkta Survey on a new AdWords Ad, you can trial over thousands of variable in one survey. Predikkta’s methods are much more efficient, and it is much faster and easier to identify customer trends.
Thereafter, you can continue to A/B Test the AdWords campaign to ensure the optimum Click-Through Rate. A Predikkta Survey may show consumer patterns, which can be explored via A/B testing.
Predikkta created the SERP Optimizer and AdWords Optimizer to save time and be more efficient.
Predictive Analytics vs A/B Testing
Testing: Numerous vs. one variable
As has been noted, a key difference between Predictive Analytics and A/B Testing is that with Predictive Analytics, you can test all possible variations and combinations in one discrete survey, whereas with A/B Testing, you can only test one variable at a time. With A/B Testing, you are forced to discount every other variable to influence a decision notwithstanding they are relevant variables in the process.
Simulated vs. Live
A/B tests are run in a live environment where visitors are randomly assigned to one of the two variations. Testing in a real environment has the disadvantage that every tweak you apply can have an unpredictable (maybe bad, or maybe good) influence on your business, without noticing it first.
Real world data has really only one benefit over data from a well-designed experiment; it is what actually happened. In every other way, real-world data is weaker.
Using data from experiments ensures that biases are controlled. Unlikely, but important events can be tested; naturally linked factors such as price and quality can be separated and independently measured and the experiment can be repeated.
With Predictive Analytics, you can run simulated 'what-if' experiments. Experiments that can test the effectiveness of the future impact of strategic options. Experiments that will support better decision-making.Testing in a simulated environment has the advantage that you can test substantial or subtle changes without losing money, customers or business credibility. Learn by foresight instead of trial and error.
Time: One test vs. Years of testing
A full SERP and AdWords Optimizer survey has over 4,000 possible outcomes and all those possibilities can be tested in hours. The Nobel Prize algorithm behind the Optimizer Tools conducts the analysis on a subset of all possibilities and therefore needs a significant lower amount of responses to give significant results. The SERP and AdWords Optimizer can give you results within hours while A/B Testing can be time consuming, as it heavily depends on your website traffic. On an A/B test, how long would it take to perform 4,000 tests?
The optimum outcome
A/B Testing or Split Testing answers the question: "How much better does one variable perform compared to the other?" It doesn't take into account that other combinations are possible and that results can be biased due to other factors. Predictive Analytics on the other hand answers 'which one' is the best combination, and takes into account the possible variations.
A solution for A/B Testing is to conduct multiple split tests. However, none of the information from previous tests can be reused to draw conclusions about the other variable you may want to test in the future. Because you can focus on only one change at a time, you may forget about other important elements that you like to test.
It the combination of both A/B Testing and Predictive Analytics, therefore, that is the most powerful combination.
Efficiency
You can determine from one predictive data analysis test 'which' variation works best. Due to the limited scope, A/B Testing can only test one change at a time. Conducting multiple split tests back-to-back is wasteful data collection. None of the information from a previous test can be reused to draw conclusions about the other variables you may want to test in the future.
A common alternative to A/B Testing is Multivariate Testing.
What is Multivariate Testing?
Where A/B tests show the effect of a single variable, Multivariate Testing can show the effect of more than one variable. This is done by systematic controlled variation of the variables of interest. The superiority of Multivariate Testing over A/B Testing is that cross-comparison of each variable can be made and some will be more important than others. Multivariate Testing is better science than A/B Testing.
Back to the washing powder example: With Multivariate Testing, you can test different variables together, like different fabrics, water temperature, etc.
But how does Multivariate Testing compare to Predictive Analytics?
Predictive Analytics vs. Multivariate Testing
Predictive Analytics includes Multivariate Testing as one of the tools in the toolbox. Predictive Analytics and Multivariate Testing are comparable but different:
Multivariate Testing is complex and needs a lot of data/traffic before you can conclude significant results. The results of multivariate analysis are not always easy to interpret and tend to be based on assumptions. A common mistake marketers make is interpreting results of a Multivariate Test on too small of z sample size, as generating enough data/traffic takes too long.
For example, if you want to conduct a multivariate test on five (5) variables in your AdWords campaign, the test consists of over 3,000 possible combinations, so given the methodology of testing, you will need a sample size in excess of 15,000. For many, the amount of website traffic you need to get significant results from multivariate testing is simply too much.
So Multivariate Testing is good science but on a practical level, it requires too much traffic. The practical solution is Predictive Analytics.
With Predictive Analytics, generally the total population is divided in different categories. Each person gets a different survey and the Nobel Prize winning algorithm takes a few samples from each category, providing a result that conclusions can be drawn from on a relative small sample size, with greater accuracy.
Governments and some of the largest companies in the world use Predictive Analytics to make billion dollar decisions. There are not enough people on the planet to multivariate test scenarios in most situations.
Apply the best available science, Predictive Analytics, to kick start your Search Result and find the optimum solution to increase website traffic.
