nnrl/nn/utils.py
def perturb_params(target: nn.Module, origin: nn.Module, stddev: float):
    """Set the parameters of a module to a noisy version of another's.

    Loads state dict from the origin module and perturbs the parameters of the
    target module. Layer normalization parameters are ignored.

    Args:
        target: the module to perturb
        origin: the module to copy from
        stddev: the gaussian standard deviation of the noise added to the origin
            model's parameters
    """
    target.load_state_dict(origin.state_dict())

    layer_norms = (m for m in target.modules() if isinstance(m, nn.LayerNorm))
    layer_norm_params = set(p for m in layer_norms for p in m.parameters())
    to_perturb = (p for p in target.parameters() if p not in layer_norm_params)

    for param in to_perturb:
        param.data.add_(torch.randn_like(param) * stddev)
Set the parameters of a module to a noisy version of another's.

    Loads state dict from the origin module and perturbs the parameters of the
    target module. Layer normalization parameters are ignored.

    Args:
        target: the module to perturb
        origin: the module to copy from
        stddev: the gaussian standard deviation of the noise added to the origin
            model's parameters
    
Set the parameters of a module to a noisy version of another's.
Loads state dict from the origin module and perturbs the parameters of the
target module. Layer normalization parameters are ignored.
Set the parameters of a module to a noisy version of another ' s . Loads state dict from the origin module and perturbs the parameters of the target module . Layer normalization parameters are ignored .
def perturb_params(target: nn.Module, origin: nn.Module, stddev: float):
    target.load_state_dict(origin.state_dict())
    layer_norms = (m for m in target.modules() if isinstance(m, nn.LayerNorm))
    layer_norm_params = set(p for m in layer_norms for p in m.parameters())
    to_perturb = (p for p in target.parameters() if p not in layer_norm_params)
    for param in to_perturb:
        param.data.add_(torch.randn_like(param) * stddev)
def perturb_params ( target : nn . Module , origin : nn . Module , stddev : float ) : target . load_state_dict ( origin . state_dict ( ) ) layer_norms = ( m for m in target . modules ( ) if isinstance ( m , nn . LayerNorm ) ) layer_norm_params = set ( p for m in layer_norms for p in m . parameters ( ) ) to_perturb = ( p for p in target . parameters ( ) if p not in layer_norm_params ) for param in to_perturb : param . data . add_ ( torch . randn_like ( param ) * stddev )
Set the parameters of a module to a noisy version of another's.
Set the parameters of a module to a noisy version of another ' s .
"""Set the parameters of a module to a noisy version of another's.

    Loads state dict from the origin module and perturbs the parameters of the
    target module. Layer normalization parameters are ignored.

    Args:
        target: the module to perturb
        origin: the module to copy from
        stddev: the gaussian standard deviation of the noise added to the origin
            model's parameters
    """
target
nn.Module
the module to perturb
the module to perturb
origin
nn.Module
the module to copy from
the module to copy from
stddev
float
the gaussian standard deviation of the noise added to the origin
model's parameters
the gaussian standard deviation of the noise added to the origin model ' s parameters