One statistical concept that Internet marketers often overlook is the difference between correlation and causation. Understanding the difference between the two will allow stakeholders to make a decision on whether to adopt changes site wide or not. Once Internet Marketers have tested a hypothesis with variation pages that included several test elements, the confirmation on whether the test elements were influential in increasing conversion should be confirmed.
We created four variation pages based on two test elements. As you can see, Variation 2 and Variation 4 are the top two performing pages, contributed by elements H1, C2, H2. The only common element in the top performing variation pages is C2 or Call to Action – 2. This is where understanding the difference between correlation and causation is crucial.
Did Call to Action -2 contribute towards the increase in conversion?
Correlation in CRO means that the test element and the conversion rate have some kind of a relationship.
But based on two variation pages, you cannot conclude that Call to Action -2 caused the increase in conversion rate.
Call to Action-2 contributing towards the increase in conversion rate can only be implied in a true experiment.
So what is true experiment in Conversion Rate Optimization?
When Internet Marketers have complete control over at least one element of the experiment, then the Call to Action-2 element can be randomly placed in each experiment. Let us say that we are testing two similar pages with Call to Action -2 and the marketer has control over traffic allocation by Country of visitor. So when a visitor from US reaches your website, she is randomly allocated to Page 1 or 2. Now we measure the conversion rate for visitors from US. If we get a similar conversion rate above the actual control with +/- 2% variation, Call to Action-2 caused the increase in conversion.