class_docstr: Module that implements a linear layer with block matrices with positive diagonal
  blocks. Moreover, it uses Weight Normalization
  (https://arxiv.org/abs/1602.07868) for stability.
class_docstr_tok: ["Module","that","implements","a","linear","layer","with","block","matrices","with","positive","diagonal","blocks",".","Moreover",",","it","uses","Weight","Normalization","(","https",":","\/\/arxiv.org\/abs\/1602.07868",")","for","stability","."]
docstr: Computes the weight matrix using masks and weight normalization.
  It also compute the log diagonal blocks of it.
docstr_tok: ["Computes","the","weight","matrix","using","masks","and","weight","normalization",".","It","also","compute","the","log","diagonal","blocks","of","it","."]
code: def get_weights(self):
      """
      Computes the weight matrix using masks and weight normalization.
      It also compute the log diagonal blocks of it.
      """
  
      # Form block weight matrix, making sure it's positive on diagonal!
      w = (
          torch.exp(self._weight) * self.mask_d
          + self._weight * self.mask_o
      )
  
      # Sum is taken over columns, i.e. one norm per row
      w_squared_norm = (w**2).sum(-1, keepdim=True)
  
      # Effect of multiplication and division is that each row is normalized and rescaled
      w = self._diag_weight.exp() * w / (w_squared_norm.sqrt() + eps)
  
      # Taking the effect of weight normalization into account in calculating the log-gradient is straightforward!
      # Instead of differentiating, e.g. d(W_1x)/dx, we have d(g_1W_1/(W_1^TW_1)^0.5x)/dx, roughly speaking, and
      # taking the log gives the right hand side below:
      wpl = (
          self._diag_weight
          + self._weight
          - 0.5 * torch.log(w_squared_norm + eps)
      )
  
      return w, wpl[self.mask_d.bool()].view(
          self.dim,
          self.out_features // self.dim,
          self.in_features // self.dim,
      )
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
  from ..None import constraints
  from ..torch_transform import TransformModule
  from ..util import copy_docs_from
  from .neural_autoregressive import ELUTransform, LeakyReLUTransform, TanhTransform
label_desc: Use image node_id 2 for calling the MaskedBlockLinear obj's underlying member method code with example usage: obj.get_weights() and returns: w,  wpl