How should dropout change with network depth?
We can think of the dropout as of averaging over many networks. Not as a way to mute or denoise your data. Therefore the rule of thumb is the dropout should be applied to higher layers in your network. It is usually applied to fully connected layers (if at all!). It is a common practice to keep 0.5-0.75. neurons active. Personally I prefer L2 much more.