class_docstr: The HybridGatedMLPContainer supports models for which the first MLP layer
  is represented with two separate weights, one for the activation function
  and one for the gating function.
class_docstr_tok: ["The","HybridGatedMLPContainer","supports","models","for","which","the","first","MLP","layer","is","represented","with","two","separate","weights",",","one","for","the","activation","function","and","one","for","the","gating","function","."]
docstr: In `set_mlp_gate`, it is necessary to populate the following variables (where appropriate)
  for the given model:
      self.inter_up_w: inter up weight
      self.inter_up_b: inter up bias
      self.inter_gate_w: inter gate weight
      self.inter_gate_b: inter gate bias
  If the parameter does not exist in the original model, set the attribute to None.
docstr_tok: ["In","`","set_mlp_gate","`",",","it","is","necessary","to","populate","the","following","variables","(","where","appropriate",")","for","the","given","model",":","self.inter_up_w",":","inter","up","weight","self.inter_up_b",":","inter","up","bias","self.inter_gate_w",":","inter","gate","weight","self.inter_gate_b",":","inter","gate","bias","If","the","parameter","does","not","exist","in","the","original","model",",","set","the","attribute","to","None","."]
code: def set_mlp_gate(self):
      """
      In `set_mlp_gate`, it is necessary to populate the following variables (where appropriate)
      for the given model:
          self.inter_up_w: inter up weight
          self.inter_up_b: inter up bias
          self.inter_gate_w: inter gate weight
          self.inter_gate_b: inter gate bias
      If the parameter does not exist in the original model, set the attribute to None.
      """
      raise NotImplementedError(
          "A set_mlp_gate() function must be defined in the model container \
                                      in order to set the unfused inter up and gate tensors."
      )
  
file_path: DeepSpeed/deepspeed/module_inject/containers/features/gated_mlp.py
imports: from abc import abstractmethod
  from .hybrid_engine import HybridEngineContainer
label_desc: Use image node_id 2 for calling the HybridGatedMLPContainer obj's underlying member method code with example usage: obj.set_mlp_gate() without return types