scripts/grad_estimators.py
def estimate_grad_vanilla(self, expl_directions, rewards):
        """
        Standard REINFORCE gradient estimator, with optional scaling by the stdev of the rewards.

        :param expl_directions: The set of perturbations.
        :param rewards_pos: Rewards obtained from perturbed models. rewards_pos[i] is the rewards of 
                            the model perturbed with expl_directions[i].
        """
        if self.use_only_best:
            sort_idx = torch.argsort(rewards, descending=True)
            rewards = rewards[sort_idx][:self.use_only_best]
            expl_directions = expl_directions[sort_idx][self.use_only_best]

        return rewards @ expl_directions / (len(rewards) * self.normalizer(rewards))

        Standard REINFORCE gradient estimator, with optional scaling by the stdev of the rewards.

        :param expl_directions: The set of perturbations.
        :param rewards_pos: Rewards obtained from perturbed models. rewards_pos[i] is the rewards of 
                            the model perturbed with expl_directions[i].
        
Standard REINFORCE gradient estimator, with optional scaling by the stdev of the rewards.
Standard REINFORCE gradient estimator with optional scaling by the stdev of the rewards .
def estimate_grad_vanilla(self, expl_directions, rewards):
        if self.use_only_best:
            sort_idx = torch.argsort(rewards, descending=True)
            rewards = rewards[sort_idx][:self.use_only_best]
            expl_directions = expl_directions[sort_idx][self.use_only_best]
        return rewards @ expl_directions / (len(rewards) * self.normalizer(rewards))
def estimate_grad_vanilla ( self , expl_directions , rewards ) : if self . use_only_best : sort_idx = torch . argsort ( rewards , descending = True ) rewards = rewards [ sort_idx ] [ : self . use_only_best ] expl_directions = expl_directions [ sort_idx ] [ self . use_only_best ] return rewards @ expl_directions / ( len ( rewards ) * self . normalizer ( rewards ) )
Standard REINFORCE gradient estimator, with optional scaling by the stdev of the rewards.
Standard REINFORCE gradient estimator with optional scaling by the stdev of the rewards .
"""
        Standard REINFORCE gradient estimator, with optional scaling by the stdev of the rewards.

        :param expl_directions: The set of perturbations.
        :param rewards_pos: Rewards obtained from perturbed models. rewards_pos[i] is the rewards of 
                            the model perturbed with expl_directions[i].
        """
self
expl_directions
The set of perturbations.
The set of perturbations .
rewards
rewards_pos
Rewards obtained from perturbed models. rewards_pos[i] is the rewards of
the model perturbed with expl_directions[i].
Rewards obtained from perturbed models . rewards_pos [ i ] is the rewards of the model perturbed with expl_directions [ i ] .