class_docstr: Call a Sparsely gated mixture of experts layer with 1-layer Feed-Forward networks as experts.
  Args:
  input_size: integer - size of the input
  output_size: integer - size of the input
  num_experts: an integer - number of experts
  hidden_size: an integer - hidden size of the experts
  noisy_gating: a boolean
  k: an integer - how many experts to use for each batch element
code: def __init__(
      self,
      input_size,
      output_size,
      num_experts,
      hidden_size,
      noisy_gating=True,
      k=4,
  ):
      super(MoE, self).__init__()
      self.noisy_gating = noisy_gating
      self.num_experts = num_experts
      self.output_size = output_size
      self.input_size = input_size
      self.hidden_size = hidden_size
      self.k = k
      # instantiate experts
      self.experts = nn.ModuleList(
          [
              MLP(self.input_size, self.output_size, self.hidden_size)
              for i in range(self.num_experts)
          ]
      )
      self.w_gate = nn.Parameter(
          torch.zeros(input_size, num_experts), requires_grad=True
      )
      self.w_noise = nn.Parameter(
          torch.zeros(input_size, num_experts), requires_grad=True
      )
  
      self.softplus = nn.Softplus()
      self.softmax = nn.Softmax(1)
      self.register_buffer("mean", torch.tensor([0.0]))
      self.register_buffer("std", torch.tensor([1.0]))
      assert self.k <= self.num_experts
  
imports: import torch
  import torch.nn
  from torch.distributions.normal import Normal
  import numpy
label_desc: Use image node_id 1 to create a new MoE object from inherited base classes: nn with example: obj = MoE(input_size, output_size, num_experts, hidden_size, noisy_gating, k)