Dropout without the averaging
With dropout, the sum of activations feeding each neuron during training and testing needs to be roughly the same. The problem is that during training some neurons are masked out, and this does not happen during testing. Therefore there has to be a scaling difference between train and test phases using dropout, otherwise each neuron would be receiving larger magnitude inputs and the network as a whole could behave very differently.

This is not mathematically the same as averaging the weights - although the effect is compared intuitively to bagging many related neural networks and averaging their output. Instead, it is usually explained that dropout scales the neuron activations to have same expected total magnitude when dropout is active (during training) vs inactive (during testing/prediction). 

Applying L2 or any other kind of parameter-controlling regularisation technique will not help with this, because these do not change any of the activations between train and test phases.

You either treat training activation values as canonical and adjust activations to be lower at test time (multiply by keep probability $p$), which is "classic dropout" or you treat test activation values as canonical and multiply activations by $\frac{1}{p}$ during training, which is called "inverse dropout". The latter approach is nowadays more common, as it is possible to completely ignore dropout layers during prediction that way.