lq/funlib.py
def opt_onestep(self, g):
        '''
        This function calculate one iteration of adam optimization. It takes the gradient of cost functin with repect to
        parameter thetha and return  dtheta. Note that you should use +dtheta when you are maximizing and -dtheta when
        minimizing.
        return the changes for the learning parameter
        :param g: Assume as gradient of loss with respect to the parameter theta
        :return: dtheta
        '''
        self.adam_M = self.beta1 * self.adam_M + (1 - self.beta1) * g
        self.adam_V = self.beta2 * self.adam_V + (1 - self.beta2) * (g * g)
        mhat = copy.copy(self.adam_M) / (1 - self.beta1 ** (self.it_index + 1))
        vhat = copy.copy(self.adam_V) / (1 - self.beta2 ** (self.it_index + 1))
        self.it_index = self.it_index + 1
        return self.step_size * mhat / (np.sqrt(vhat) + self.epsilon)

        This function calculate one iteration of adam optimization. It takes the gradient of cost functin with repect to
        parameter thetha and return  dtheta. Note that you should use +dtheta when you are maximizing and -dtheta when
        minimizing.
        return the changes for the learning parameter
        :param g: Assume as gradient of loss with respect to the parameter theta
        :return: dtheta
        
This function calculate one iteration of adam optimization. It takes the gradient of cost functin with repect to
parameter thetha and return  dtheta. Note that you should use +dtheta when you are maximizing and -dtheta when
minimizing.
return the changes for the learning parameter
This function calculate one iteration of adam optimization . It takes the gradient of cost functin with repect to parameter thetha and return dtheta . Note that you should use + dtheta when you are maximizing and - dtheta when minimizing . return the changes for the learning parameter
def opt_onestep(self, g):
        self.adam_M = self.beta1 * self.adam_M + (1 - self.beta1) * g
        self.adam_V = self.beta2 * self.adam_V + (1 - self.beta2) * (g * g)
        mhat = copy.copy(self.adam_M) / (1 - self.beta1 ** (self.it_index + 1))
        vhat = copy.copy(self.adam_V) / (1 - self.beta2 ** (self.it_index + 1))
        self.it_index = self.it_index + 1
        return self.step_size * mhat / (np.sqrt(vhat) + self.epsilon)
def opt_onestep ( self , g ) : self . adam_M = self . beta1 * self . adam_M + ( 1 - self . beta1 ) * g self . adam_V = self . beta2 * self . adam_V + ( 1 - self . beta2 ) * ( g * g ) mhat = copy . copy ( self . adam_M ) / ( 1 - self . beta1 ** ( self . it_index + 1 ) ) vhat = copy . copy ( self . adam_V ) / ( 1 - self . beta2 ** ( self . it_index + 1 ) ) self . it_index = self . it_index + 1 return self . step_size * mhat / ( np . sqrt ( vhat ) + self . epsilon )
This function calculate one iteration of adam optimization.
This function calculate one iteration of adam optimization .
'''
        This function calculate one iteration of adam optimization. It takes the gradient of cost functin with repect to
        parameter thetha and return  dtheta. Note that you should use +dtheta when you are maximizing and -dtheta when
        minimizing.
        return the changes for the learning parameter
        :param g: Assume as gradient of loss with respect to the parameter theta
        :return: dtheta
        '''
None
self
g
Assume as gradient of loss with respect to the parameter theta
Assume as gradient of loss with respect to the parameter theta