title: How to implement dropout in Pytorch, and where to apply it
body: <p>I am quite unsure whether this is correct. It is really sad I can't find many good examples on how to parametrize a NN. </p>
  
  <p>What do you think of this way of dropping out in those two classes. First I'm writing the original class :</p>
  
  <pre><code>class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes, p = dropout):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, num_classes)
  
    def forward(self, x):
        out = F.relu(self.fc1(x))
        out = F.relu(self.fc2(out))
        out = self.fc3(out)
        return out
  </code></pre>
  
  <p>and then here, I found two different ways to write things, which I don't know how to distinguish. The first one uses : </p>
  
  <pre><code>self.drop_layer = nn.Dropout(p=p) 
  </code></pre>
  
  <p>whereas the second :</p>
  
  <pre><code>self.dropout = nn.Dropout(p) 
  </code></pre>
  
  <p>and here is my result : </p>
  
  <pre><code>class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes, p = dropout):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, num_classes)
        self.drop_layer = nn.Dropout(p=p)
  
    def forward(self, x):
        out = F.relu(self.fc1(x))
        out = F.relu(self.fc2(out))
        out = self.fc3(out)
        return out
  
  
   class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes, p = dropout):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, num_classes)
        self.dropout = nn.Dropout(p) 
  
    def forward(self, x):
        out = F.relu(self.fc1(x))
        out = F.relu(self.fc2(out))
        out = self.fc3(out)
        return out
  </code></pre>
  
  <p>could this work, if not how to improve that, and is it giving me the result I m expecting, meaning creating a neural network where I can dropout some neurons. <strong>Important detail, I only want to dropout the second layer of neural network, no touch to the rest!</strong></p>