go ahead and play with thissee what the effect of different standard deviations has on the initial datasetsor differences in meansand different sample sizes just want you to dive inplay around with these different datasets and actually run themand see what the effect is on the -statistic and the -value and hopefully that will give you more gut feel of how to interpret these results againthe important thing to understand is that you're looking for large -statistic and small -value -value is probably going to be what you want to communicate to the business and rememberlower is better for -valueyou want to see that in the single digitsideally below percent before you declare victory we'll talk about / tests some more in the remainder of the scipy makes it really easy to compute -statistics and -values for given set of dataso you can very easily compare the behavior between your control and treatment groupsand measure what the probability is of that effect being real or just result of random variation make sure you are focusing on those metrics and you are measuring the conversion metric that you care about when you're doing those comparisons determining how long to run an experiment for how long do you run an experiment forhow long does it take to actually get resultat what point do you give uplet' talk about that in more detail if someone in your company has developed new experimenta new change that they want to testthen they have vested interest in seeing that succeed they put lot of work and time into itand they want it to be successful maybe you've gone weeks with the testing and you still haven' reached significant outcome on this experimentpositive or negative you know that they're going to want to keep running it pretty much indefinitely in the hope that it will eventually show positive result it' up to you to draw the line on how long you're willing to run this experiment for how do know when ' done running an / testi meanit' not always straightforward to predict how long it will take before you can achieve significant resultbut obviously if you have achieved significant resultif your -value has gone below percent or percent or whatever threshold you've chosenand you're done