rfho/hyper_gradients.py
def run_all(self, T, train_feed_dict_supplier=None, val_feed_dict_suppliers=None, hyper_batch_step=None,
                forward_su=None, backward_su=None, after_forward_su=None, check_if_zero=False):
        """
        Performs both forward and backward step. See functions `forward` and `backward` for details.

        :param hyper_batch_step: support for stochastic sampling of validation set
        :param T:                   Total number of iterations
        :param train_feed_dict_supplier:   (feed_dict) supplier for training stage
        :param val_feed_dict_suppliers: (feed_dict) supplier for validation stage
        :param forward_su:          (optional) utils object with function `run` passed to `forward`
        :param backward_su:         (optional) utils object with function `run` passed to `backward`
        :param after_forward_su:    (optional) utils object with function `run` executed after `forward` and before
                                    `backward`
        :param check_if_zero:       (debug flag).
        :return: A dictionary of lists of step-wise hyper-gradients. In usual application the "true" hyper-gradients
                 can be obtained with method `std_collect_hyper_gradients`
        """
        # self.initialize()
        self.forward(T, train_feed_dict_supplier=train_feed_dict_supplier, summary_utils=forward_su)
        final_w = self.w_t.eval()
        last_global_step = self.global_step.eval()

        if after_forward_su:
            after_forward_su.run(tf.get_default_session(), T)

        raw_hyper_grads = self.backward(
            T, val_feed_dict_suppliers=val_feed_dict_suppliers,
            train_feed_dict_supplier=train_feed_dict_supplier, summary_utils=backward_su,
            check_if_zero=check_if_zero,
            hyper_batch_step=hyper_batch_step
        )

        tf.get_default_session().run(self._back_hist_op,
                                     feed_dict={self._w_placeholder: final_w})  # restore weights

        tf.get_default_session().run(self.global_step.assign_op,
                                     feed_dict={self.global_step.gs_placeholder: last_global_step})

        return raw_hyper_grads

        Performs both forward and backward step. See functions `forward` and `backward` for details.

        :param hyper_batch_step: support for stochastic sampling of validation set
        :param T:                   Total number of iterations
        :param train_feed_dict_supplier:   (feed_dict) supplier for training stage
        :param val_feed_dict_suppliers: (feed_dict) supplier for validation stage
        :param forward_su:          (optional) utils object with function `run` passed to `forward`
        :param backward_su:         (optional) utils object with function `run` passed to `backward`
        :param after_forward_su:    (optional) utils object with function `run` executed after `forward` and before
                                    `backward`
        :param check_if_zero:       (debug flag).
        :return: A dictionary of lists of step-wise hyper-gradients. In usual application the "true" hyper-gradients
                 can be obtained with method `std_collect_hyper_gradients`
        
Performs both forward and backward step. See functions `forward` and `backward` for details.
Performs both forward and backward step . See functions ` forward ` and ` backward ` for details .
def run_all(self, T, train_feed_dict_supplier=None, val_feed_dict_suppliers=None, hyper_batch_step=None,
                forward_su=None, backward_su=None, after_forward_su=None, check_if_zero=False):
        self.forward(T, train_feed_dict_supplier=train_feed_dict_supplier, summary_utils=forward_su)
        final_w = self.w_t.eval()
        last_global_step = self.global_step.eval()
        if after_forward_su:
            after_forward_su.run(tf.get_default_session(), T)
        raw_hyper_grads = self.backward(
            T, val_feed_dict_suppliers=val_feed_dict_suppliers,
            train_feed_dict_supplier=train_feed_dict_supplier, summary_utils=backward_su,
            check_if_zero=check_if_zero,
            hyper_batch_step=hyper_batch_step
        )
        tf.get_default_session().run(self._back_hist_op,
                                     feed_dict={self._w_placeholder: final_w})  
        tf.get_default_session().run(self.global_step.assign_op,
                                     feed_dict={self.global_step.gs_placeholder: last_global_step})
        return raw_hyper_grads
def run_all ( self , T , train_feed_dict_supplier = None , val_feed_dict_suppliers = None , hyper_batch_step = None , forward_su = None , backward_su = None , after_forward_su = None , check_if_zero = False ) : self . forward ( T , train_feed_dict_supplier = train_feed_dict_supplier , summary_utils = forward_su ) final_w = self . w_t . eval ( ) last_global_step = self . global_step . eval ( ) if after_forward_su : after_forward_su . run ( tf . get_default_session ( ) , T ) raw_hyper_grads = self . backward ( T , val_feed_dict_suppliers = val_feed_dict_suppliers , train_feed_dict_supplier = train_feed_dict_supplier , summary_utils = backward_su , check_if_zero = check_if_zero , hyper_batch_step = hyper_batch_step ) tf . get_default_session ( ) . run ( self . _back_hist_op , feed_dict = { self . _w_placeholder : final_w } ) tf . get_default_session ( ) . run ( self . global_step . assign_op , feed_dict = { self . global_step . gs_placeholder : last_global_step } ) return raw_hyper_grads
Performs both forward and backward step.
Performs both forward and backward step .
"""
        Performs both forward and backward step. See functions `forward` and `backward` for details.

        :param hyper_batch_step: support for stochastic sampling of validation set
        :param T:                   Total number of iterations
        :param train_feed_dict_supplier:   (feed_dict) supplier for training stage
        :param val_feed_dict_suppliers: (feed_dict) supplier for validation stage
        :param forward_su:          (optional) utils object with function `run` passed to `forward`
        :param backward_su:         (optional) utils object with function `run` passed to `backward`
        :param after_forward_su:    (optional) utils object with function `run` executed after `forward` and before
                                    `backward`
        :param check_if_zero:       (debug flag).
        :return: A dictionary of lists of step-wise hyper-gradients. In usual application the "true" hyper-gradients
                 can be obtained with method `std_collect_hyper_gradients`
        """ # self.initialize() # restore weights
A dictionary of lists of step-wise hyper-gradients. In usual application the "true" hyper-gradients
can be obtained with method `std_collect_hyper_gradients`
A dictionary of lists of step - wise hyper - gradients . In usual application the " true " hyper - gradients can be obtained with method ` std_collect_hyper_gradients `
self
T
Total number of iterations
Total number of iterations
train_feed_dict_supplier
(feed_dict) supplier for training stage
( feed_dict ) supplier for training stage
val_feed_dict_suppliers
(feed_dict) supplier for validation stage
( feed_dict ) supplier for validation stage
hyper_batch_step
support for stochastic sampling of validation set
support for stochastic sampling of validation set
forward_su
(optional) utils object with function `run` passed to `forward`
( optional ) utils object with function ` run ` passed to ` forward `
backward_su
(optional) utils object with function `run` passed to `backward`
( optional ) utils object with function ` run ` passed to ` backward `
after_forward_su
(optional) utils object with function `run` executed after `forward` and before
`backward`
( optional ) utils object with function ` run ` executed after ` forward ` and before ` backward `
check_if_zero
None