Correct order of operations involved into Dropout
The usual processing for your suggested layers:

model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))


would be (reading left to right)

dense output -&gt; relu -&gt; apply dropout mask -&gt; apply "inverse dropout" divide by p  


The precise combination may vary depending upon optimisations, and can in theory be changed a little without affecting the result (it doesn't matter to the end result numerically if we scale then mask or mask then scale for instance). However when dealing with vectorised optimisations (like those found in TensorFlow and Theano), it is normal to accept a percentage of "wasted" processing, and just have that naive left-to-right processing happen. It is often harder to parallelise decision branches than to simply process all items, even repeated multiplying by and adding zeroes for a significant fraction of each array.

There is no "normalization" implied by activation functions, so this is not a concern. From your comments, it seems you are worried that dividing by p could mean that the output of a neuron that would be between 0 and 1 (because you were using sigmoid activation for example) would now be between 0 and 1/p - i.e. larger. That is true. Is this a problem? No it is not, and in fact it is required. The impact of the larger values is fully compensated for by the weights learned in the connections between layers. If you used "vanilla" dropout then the weights would be correspondingly larger, but you would need to scale the outputs down (new range would be 0 to p) during testing/prediction.