at that point you can pull the plug on the experiment and either roll out the change more widely or remove it because it was actually having negative effect you can always tell people to go back and try againuse what they learned from the experiment to maybe try it again with some changes and soften the blow little bit the other thing that might happen is it' just not converging at all if you're not seeing any trends over time in the -valueit' probably good sign that you're not going to see this converge anytime soon it' just not going to have enough of an impact on behavior to even be measurableno matter how long you run it in those situationswhat you want to do every day is plot on graph for given experiment the -valuethe -statisticwhatever you're using to measure the success of this experimentand if you're seeing something that looks promisingyou will see that -value start to come down over time sothe more data it getsthe more significant your results should be getting nowif you instead see flat line or line that' all over the placethat kind of tells you that that -value' not going anywhereand it doesn' matter how long you run this experimentit' just not going to happen you need to agree up front that in the case where you're not seeing any trends in -valueswhat' the longest you're willing to run this experiment foris it two weeksis it monthanother thing to keep in mind is that having more than one experiment running on the site at once can conflate your results time spent on experiments is valuable commodityyou can' make more time in the world you can only really run as many experiments as you have time to run them in given year soif you spend too much time running one experiment that really has no chance of converging on resultthat' an opportunity you've missed to run another potentially more valuable experiment during that time that you are wasting on this other one it' important to draw the line on experiment linksbecause time is very precious commodity when you're running / tests on websiteat least as long as you have more ideas than you have timewhich hopefully is the case make sure you go in with agreed upper bounds on how long you're going to spend testing given experimentand if you're not seeing trends in the -value that look encouragingit' time to pull the plug at that point