How can we effectively measure the impact of our data decisions
In A/B testing, bias is handled very well by ensuring visitors are randomly assigned to either version A or version B of the site.  This creates independent samples drawn from the same population.  Because the groups are independent and, on average, only differ in the version of the site seen, the test measures the effect of the design decision.

Slight aside: Now you might argue that the A group or B group may differ in some demographic.  That commonly happens by random chance.  To a certain degree this can be taken care of by covariate adjusted randomization.  It can also be taken care of by adding covariates to the model that tests the effect of the design decision.  It should be noted that there is still some discussion about the proper way to do this within the statistics community.  Essentially A/B testing is an application of a Randomized Control Trial to website design.  Some people disagree with adding covariates to the test.  Others, such as Frank Harrel (see Regression Modeling Strategies) argue for the use of covariates in such models.

I would offer the following suggestions:


Design the study in advance so as to take care of as much sources of bias and variation as possible.   
Let the data speak for itself.  As you get more data (like about searches for David Beckham), let it dominate your assumptions about how the data should be (as how the posterior dominates the prior in Bayesian analysis when the sample size becomes large).    
Make sure your data matches the assumptions of the model.