measuring conversion for / testing the first thing you need to figure out when you're designing an experiment on website is what are you trying to optimize forwhat is it that you really want to drive with this changeand this isn' always very obvious thing maybe it' the amount that people spendthe amount of revenue wellwe talked about the problems with variance in using amount spentbut if you have enough datayou can stillreach convergence on that metric lot of times howevermaybe that' not what you actually want to optimize for maybe you're actually selling some items at loss intentionally just to capture market share there' more complexity that goes into your pricing strategy than just top-line revenue maybe what you really want to measure is profitand that can be very tricky thing to measurebecause lot of things cut into how much money given product might make and those things might not always be obvious and againif you have loss leadersthis experiment will discount the effect that those are supposed to have maybe you just care about driving ad clicks on your websiteor order quantities to reduce variancemaybe people are okay with that the bottom line is that you have to talk to the business owners of the area that' being tested and figure out what it is they're trying to optimize for what are they being measured onwhat is their success measured onwhat are their key performance indicators or whatever the nbas want to call itand make sure that we're measuring the thing that it matters to them you can measure more than one thing at once tooyou don' have to pick oneyou can actually report on the effect of many different thingsrevenue profit clicks ad views if these things are all moving in the right direction togetherthat' very strong sign that this change had positive impact in more ways than one sowhy limit yourself to one metricjust make sure you know which one matters the most in what' going to be your criteria for success of this experiment ahead of time