models/dense_net.py
def self_constr_rlr1(self, learning_rate, initial_lr, rlr_1, rlr_2):
        """
        An optional learning rate reduction (Reduce LR #1) to be performed
        after a step of the self-constructing algorithm (based on the patience
        countdown, so it only works with variant #2 onwards).
        Returns the new learning rate value.

        The initial learning rate value is initial_lr.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_1), the learning rate becomes initial_lr/10.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_2), the learning rate becomes initial_lr/100.

        Args:
            learning_rate: `int`, the current learning rate value.
            initial_lr: the initial value for the learning rate.
            rlr_1: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the first time.
            rlr_2: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the second time.
        """
        if not self.has_micro_algo:
            patience_cntdwn = self.patience_cntdwn
            patience_param = self.patience_param
        else:
            patience_cntdwn = self.m_patience_cntdwn
            patience_param = self.m_patience_param

        if (patience_cntdwn == int(patience_param * (1-rlr_1))):
            learning_rate = min(learning_rate, initial_lr / 10)
        elif (patience_cntdwn == int(patience_param * (1-rlr_2))):
            # learning_rate = min(learning_rate, initial_lr / 100)
            learning_rate = initial_lr / 100  # min is unnecessary here
        return learning_rate

        An optional learning rate reduction (Reduce LR #1) to be performed
        after a step of the self-constructing algorithm (based on the patience
        countdown, so it only works with variant #2 onwards).
        Returns the new learning rate value.

        The initial learning rate value is initial_lr.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_1), the learning rate becomes initial_lr/10.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_2), the learning rate becomes initial_lr/100.

        Args:
            learning_rate: `int`, the current learning rate value.
            initial_lr: the initial value for the learning rate.
            rlr_1: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the first time.
            rlr_2: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the second time.
        
An optional learning rate reduction (Reduce LR #1) to be performed
after a step of the self-constructing algorithm (based on the patience
countdown, so it only works with variant #2 onwards).
Returns the new learning rate value.

The initial learning rate value is initial_lr.
The first time that the countdown reaches an epoch that corresponds to
patience_param * (1 - rlr_1), the learning rate becomes initial_lr/10.
The first time that the countdown reaches an epoch that corresponds to
patience_param * (1 - rlr_2), the learning rate becomes initial_lr/100.
An optional learning rate reduction ( Reduce LR #1 ) to be performed after a step of the self - constructing algorithm ( based on the patience countdown so it only works with variant #2 onwards ) . Returns the new learning rate value . The initial learning rate value is initial_lr . The first time that the countdown reaches an epoch that corresponds to patience_param * ( 1 - rlr_1 ) the learning rate becomes initial_lr / 10 . The first time that the countdown reaches an epoch that corresponds to patience_param * ( 1 - rlr_2 ) the learning rate becomes initial_lr / 100 .
def self_constr_rlr1(self, learning_rate, initial_lr, rlr_1, rlr_2):
        if not self.has_micro_algo:
            patience_cntdwn = self.patience_cntdwn
            patience_param = self.patience_param
        else:
            patience_cntdwn = self.m_patience_cntdwn
            patience_param = self.m_patience_param
        if (patience_cntdwn == int(patience_param * (1-rlr_1))):
            learning_rate = min(learning_rate, initial_lr / 10)
        elif (patience_cntdwn == int(patience_param * (1-rlr_2))):
            learning_rate = initial_lr / 100  
        return learning_rate
def self_constr_rlr1 ( self , learning_rate , initial_lr , rlr_1 , rlr_2 ) : if not self . has_micro_algo : patience_cntdwn = self . patience_cntdwn patience_param = self . patience_param else : patience_cntdwn = self . m_patience_cntdwn patience_param = self . m_patience_param if ( patience_cntdwn == int ( patience_param * ( 1 - rlr_1 ) ) ) : learning_rate = min ( learning_rate , initial_lr / 10 ) elif ( patience_cntdwn == int ( patience_param * ( 1 - rlr_2 ) ) ) : learning_rate = initial_lr / 100 return learning_rate
An optional learning rate reduction (Reduce LR #1) to be performed
after a step of the self-constructing algorithm (based on the patience
countdown, so it only works with variant #2 onwards).
An optional learning rate reduction ( Reduce LR #1 ) to be performed after a step of the self - constructing algorithm ( based on the patience countdown so it only works with variant #2 onwards ) .
"""
        An optional learning rate reduction (Reduce LR #1) to be performed
        after a step of the self-constructing algorithm (based on the patience
        countdown, so it only works with variant #2 onwards).
        Returns the new learning rate value.

        The initial learning rate value is initial_lr.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_1), the learning rate becomes initial_lr/10.
        The first time that the countdown reaches an epoch that corresponds to
        patience_param * (1 - rlr_2), the learning rate becomes initial_lr/100.

        Args:
            learning_rate: `int`, the current learning rate value.
            initial_lr: the initial value for the learning rate.
            rlr_1: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the first time.
            rlr_2: the fraction of epochs through the countdown at which
                the learning rate must be reduced (/10) for the second time.
        """ # learning_rate = min(learning_rate, initial_lr / 100) # min is unnecessary here
self
learning_rate
`int`, the current learning rate value.
` int ` the current learning rate value .
initial_lr
the initial value for the learning rate.
the initial value for the learning rate .
rlr_1
the fraction of epochs through the countdown at which
the learning rate must be reduced (/10) for the first time.
the fraction of epochs through the countdown at which the learning rate must be reduced ( / 10 ) for the first time .
rlr_2
the fraction of epochs through the countdown at which
the learning rate must be reduced (/10) for the second time.
the fraction of epochs through the countdown at which the learning rate must be reduced ( / 10 ) for the second time .