Dropout in a CNN vs Dropout in a FCNN In the PyTorch nn module there are 2 types of dropouts:


A normal Dropout - During training, randomly zeroes some of the elements of the input tensor with probability p using samples from a Bernoulli distribution. Each channel will be zeroed out independently on every forward call.
A Dropout2d - Randomly zero out entire channels. Each channel will be zeroed out independently on every forward call with probability p using samples from a Bernoulli distribution.


From Documentation



My question is when we use CNN, which of the dropout we use in the Convolutional layers. In this paper it is being said we use the first kind. But I am not so sure what is the general practice.
Check this paper:

https://arxiv.org/pdf/1411.4280.pdf

In section 3.2 they discuss in detail why standard dropout fails in convolutional layers and the idea of spatial dropout. Fully connected networks learn via connections between single neurons while convolutions learn via features. In a sense, dropping out entire features instead of single neurons can be thought as the convolutional version of regular dropout.