neural networks and deep learning for inp in range(self num_inputs+ )gradient self delta[out][inpself nn batch_size self ms[out][inpself rho*self ms[out][inp]( -self rhogradient** self weights[out][inp-self nn learning_rate/(self ms[out][inp]+self epsilon)** gradient self delta[out][inp dropout dropout is implemented as layer learnnn py -(continued from utilities import flip class dropout_layer(layer)"""dropout layer "" def __init__(selfnnrate= )""rate is fraction of the input units to drop =rate ""self rate rate layer __init__(selfnn def output_values(selfinput_valuestraining=false)"""returns the outputs for the input values it remembers the input values for the backprop ""if trainingscaling /( -self rateself mask [ if flip(self rateelse for in input_valuesreturn [ * *scaling for ( ,yin zip(input_valuesself mask)elsereturn input_values def backprop(self,errors)"""returns the derivative of the errors""return [ * for ( ,yin zip(errorsself mask) class dropout_layer_ (layer)"""dropout layer "" def __init__(selfnnrate= )""rate is fraction of the input units to drop =rate ""version june