class_docstr: This class implements The HyperNetwork. It is an approach of using a one network,
  also known as a hypernetwork, to generate the weights for another network.
  Here, it is used to generate the labels of linear layers.
  
  Reference: https://arxiv.org/abs/1609.09106
  
  Arguments
  ----------
  input_output_dim : int
      Dimension of the linear layers
  hypernet_size:
      Dimension of the HyperNetwork
  tied : bool, optional
      Define whether weights of layer 1 and layer 2 are shared
  num_heads: int, optional
      Number of heads, akin to heads in MultiHeadAttention
  keep_output_size: bool, optional
      Set whether to keep the same output size independent of number of heads
class_docstr_tok: ["This","class","implements","The","HyperNetwork",".","It","is","an","approach","of","using","a","one","network",",","also","known","as","a","hypernetwork",",","to","generate","the","weights","for","another","network",".","Here",",","it","is","used","to","generate","the","labels","of","linear","layers",".","Reference",":","https",":","\/\/arxiv.org\/abs\/1609.09106","Arguments","--","--","--","--","--","input_output_dim",":","int","Dimension","of","the","linear","layers","hypernet_size",":","Dimension","of","the","HyperNetwork","tied",":","bool",",","optional","Define","whether","weights","of","layer","1","and","layer","2","are","shared","num_heads",":","int",",","optional","Number","of","heads",",","akin","to","heads","in","MultiHeadAttention","keep_output_size",":","bool",",","optional","Set","whether","to","keep","the","same","output","size","independent","of","number","of","heads"]
docstr: Forward computation for a HyperNetwork.
  
  Arguments
  ----------
  input_tensor : [batchsize, max_positions, d]
      The HyperNetwork is supposed to generate an MLP of the form W_2(GELU(W1 x)), where
      W1 : N -> k and W2 : k -> N, so it has to return tensors W1 and W2
  
  Outputs
  -------
  W1 : torch.Tensor
      Generated weights of Layer 1
  W2 : torch.Tensor
      Generated weights of Layer 2
docstr_tok: ["Forward","computation","for","a","HyperNetwork",".","Arguments","--","--","--","--","--","input_tensor",":","[","batchsize",",","max_positions",",","d","]","The","HyperNetwork","is","supposed","to","generate","an","MLP","of","the","form","W_2","(","GELU","(","W1","x",")",")",",","where","W1",":","N","-",">","k","and","W2",":","k","-",">","N",",","so","it","has","to","return","tensors","W1","and","W2","Outputs","--","--","--","-","W1",":","torch.Tensor","Generated","weights","of","Layer","1","W2",":","torch.Tensor","Generated","weights","of","Layer","2"]
code: def forward(self, input_tensor: torch.Tensor):
      """Forward computation for a HyperNetwork.
  
      Arguments
      ----------
      input_tensor : [batchsize, max_positions, d]
          The HyperNetwork is supposed to generate an MLP of the form W_2(GELU(W1 x)), where
          W1 : N -> k and W2 : k -> N, so it has to return tensors W1 and W2
  
      Outputs
      -------
      W1 : torch.Tensor
          Generated weights of Layer 1
      W2 : torch.Tensor
          Generated weights of Layer 2
      """
      W1 = self.w1_gen(input_tensor)
      if self.tied:
          W2 = W1
      else:
          W2 = self.w2_gen(input_tensor)
  
      return W1, W2
  
file_path: speechbrain/speechbrain/lobes/models/transformer/hypermixing.py
imports: from typing import Optional
  import math
  import torch
  from torch import nn
label_desc: Use image node_id 2 for calling the HyperNetwork obj's underlying member method code with example usage: obj.forward(input_tensor) and returns: W1,  W2