class_docstr: Helper for implementing a mixture of experts.
  The purpose of this class is to create input minibatches for the
  experts and to combine the results of the experts to form a unified
  output tensor.
  There are two functions:
  dispatch - take an input Tensor and create input Tensors for each expert.
  combine - take output Tensors from each expert and form a combined output
    Tensor.  Outputs from different experts for the same batch element are
    summed together, weighted by the provided "gates".
  The class is initialized with a "gates" Tensor, which specifies which
  batch elements go to which experts, and the weights to use when combining
  the outputs.  Batch element b is sent to expert e iff gates[b, e] != 0.
  The inputs and outputs are all two-dimensional [batch, depth].
  Caller is responsible for collapsing additional dimensions prior to
  calling this class and reshaping the output to the original shape.
  See common_layers.reshape_like().
  Example use:
  gates: a float32 `Tensor` with shape `[batch_size, num_experts]`
  inputs: a float32 `Tensor` with shape `[batch_size, input_size]`
  experts: a list of length `num_experts` containing sub-networks.
  dispatcher = SparseDispatcher(num_experts, gates)
  expert_inputs = dispatcher.dispatch(inputs)
  expert_outputs = [experts[i](expert_inputs[i]) for i in range(num_experts)]
  outputs = dispatcher.combine(expert_outputs)
  The preceding code sets the output for a particular example b to:
  output[b] = Sum_i(gates[b, i] * experts[i](inputs[b]))
  This class takes advantage of sparsity in the gate matrix by including in the
  `Tensor`s for expert i only the batch elements for which `gates[b, i] > 0`.
docstr: Sum together the expert output, weighted by the gates.
  The slice corresponding to a particular batch element `b` is computed
  as the sum over all experts `i` of the expert output, weighted by the
  corresponding gate values.  If `multiply_by_gates` is set to False, the
  gate values are ignored.
  Args:
    expert_out: a list of `num_experts` `Tensor`s, each with shape
      `[expert_batch_size_i, <extra_output_dims>]`.
    multiply_by_gates: a boolean
  Returns:
    a `Tensor` with shape `[batch_size, <extra_output_dims>]`.
docstr_tok: ["Sum","together","the","expert","output",",","weighted","by","the","gates",".","The","slice","corresponding","to","a","particular","batch","element","`","b","`","is","computed","as","the","sum","over","all","experts","`","i","`","of","the","expert","output",",","weighted","by","the","corresponding","gate","values",".","If","`","multiply_by_gates","`","is","set","to","False",",","the","gate","values","are","ignored",".","Args",":","expert_out",":","a","list","of","`","num_experts","`","`","Tensor","`","s",",","each","with","shape","`","[","expert_batch_size_i",",","<","extra_output_dims",">","]","`",".","multiply_by_gates",":","a","boolean","Returns",":","a","`","Tensor","`","with","shape","`","[","batch_size",",","<","extra_output_dims",">","]","`","."]
code: def combine(self, expert_out, multiply_by_gates=True):
      """Sum together the expert output, weighted by the gates.
      The slice corresponding to a particular batch element `b` is computed
      as the sum over all experts `i` of the expert output, weighted by the
      corresponding gate values.  If `multiply_by_gates` is set to False, the
      gate values are ignored.
      Args:
        expert_out: a list of `num_experts` `Tensor`s, each with shape
          `[expert_batch_size_i, <extra_output_dims>]`.
        multiply_by_gates: a boolean
      Returns:
        a `Tensor` with shape `[batch_size, <extra_output_dims>]`.
      """
      # apply exp to expert outputs, so we are not longer in log space
      stitched = torch.cat(expert_out, 0)
  
      if multiply_by_gates:
          stitched = stitched.mul(self._nonzero_gates)
      zeros = torch.zeros(
          self._gates.size(0),
          expert_out[-1].size(1),
          requires_grad=True,
          device=stitched.device,
      )
      # combine samples that have been processed by the same k experts
      combined = zeros.index_add(0, self._batch_index, stitched.float())
      return combined
  
imports: import torch
  import torch.nn
  from torch.distributions.normal import Normal
  import numpy
label_desc: Use image node_id 3 for calling the SparseDispatcher obj's underlying member method code with example usage: obj.combine(expert_out, multiply_by_gates) and returns: combined