the -statistic or -test let' start with the -statisticalso known as -test it is basically measure of the difference in behavior between these two setsbetween your control and treatment groupexpressed in units of standard error it is based on standard errorwhich accounts for the variance inherent in the data itselfso by normalizing everything by that standard errorwe get some measure of the change in behavior between these two groups that takes that variance into account the way to interpret -statistic is that high -value means there' probably real difference between these two setswhereas low -value means not so much difference you have to decide what' threshold that you're willing to acceptthe sign of the tstatistic will tell you if it' positive or negative change if you're comparing your control to your treatment group and you end up with negative tstatisticthat implies that this is bad change you ultimate want the absolute value of that -statistic to be large how large value of -statistic is considered largewellthat' debatable we'll look at some examples shortly nowthis does assume that you have normal distribution of behaviorand when we're talking about things like the amount people spend on websitethat' usually decent assumption there does tend to be normal distribution of how much people spend howeverthere are more refined versions of -statistics that you might want to look at for other specific situations for examplethere' something called fisher' exact test for when you're talking about click through ratesthe -test when you're talking about transactions per userlike how many web pages do they seeand the chi-squared testwhich is often relevant for when you're looking at order quantities sometimes you'll want to look at all of these statistics for given experimentand choose the one that actually fits what you're trying to do the best the -value nowit' lot easier to talk about -values than -statistics because you don' have to think abouthow many standard deviations are we talking aboutwhat does the actual value meanthe -value is little bit easier for people to understandwhich makes it better tool for you to communicate the results of an experiment to the stakeholders in your business