According to classical paperhttp://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdfdropout operation affects not only training step but also test step - we need to multiply all neuron output weights by probability p.But in keras library I found the following implementation of dropout operation:retain_prob = 1. - level...random_tensor = rng.binomial(x.shape, p=retain_prob, dtype=x.dtype)...x *= random_tensorx /= retain_probreturn x(see https://github.com/fchollet/keras/blob/master/keras/backend/theano_backend.py)Why x is divided by retain_prob while it should be multiplied?Or I'm just confused, and multiplying weights is equivelent to dividing output value?
Convolutional layer dropout layer in keras
machine learning;neural network;keras;theano;dropout
You are looking at the Keras code implementing dropout for training step. In the Keras implementation, the output values are corrected during training (by dividing, in addition to randomly dropping out the values) instead of during testing (by multiplying). This is called inverted dropout.Inverted dropout is functionally equivalent to original dropout (as per your link to Srivastava's paper), with a nice feature that the network does not use dropout layers at all during test and prediction. This is explained a little in this Keras issue.