class_docstr: This container identifies which methods need to be overridden in addition to
  the base container to enable use in the RLHF pipeline. These methods are not
  necessary for inference alone.
  
  NOTE: If you are using this feature with a container that
  also inherits from `MetaTensorContainer`, ensure that `MetaTensorContainer`
  is inherited before `HybridEngineContainer` in the class definition.
class_docstr_tok: ["This","container","identifies","which","methods","need","to","be","overridden","in","addition","to","the","base","container","to","enable","use","in","the","RLHF","pipeline",".","These","methods","are","not","necessary","for","inference","alone",".","NOTE",":","If","you","are","using","this","feature","with","a","container","that","also","inherits","from","`","MetaTensorContainer","`",",","ensure","that","`","MetaTensorContainer","`","is","inherited","before","`","HybridEngineContainer","`","in","the","class","definition","."]
docstr: Add support for reversed dim in tensor parallelism. If necessary, override
  the called methods to handle partitioned weights (i.e. if qkv is split, override
  the `attention_qkv_mp` method). If the model component is not split, it should
  be safe to use the default implementation.
docstr_tok: ["Add","support","for","reversed","dim","in","tensor","parallelism",".","If","necessary",",","override","the","called","methods","to","handle","partitioned","weights","(","i.e",".","if","qkv","is","split",",","override","the","`","attention_qkv_mp","`","method",")",".","If","the","model","component","is","not","split",",","it","should","be","safe","to","use","the","default","implementation","."]
code: def apply_tensor_parallelism(self, mp_replace, reversed_dim=False):
      """
      Add support for reversed dim in tensor parallelism. If necessary, override
      the called methods to handle partitioned weights (i.e. if qkv is split, override
      the `attention_qkv_mp` method). If the model component is not split, it should
      be safe to use the default implementation.
      """
      # Setup the new Attention module
      self.attention_qkv_mp(mp_replace, reversed_dim=reversed_dim)
      self.attention_o_mp(mp_replace, reversed_dim=reversed_dim)
  
      # Setup the new MLP module
      self.mlp_inter_mp(mp_replace, reversed_dim=reversed_dim)
      self.mlp_output_mp(mp_replace, reversed_dim=reversed_dim)
  
imports: from abc import ABC, abstractmethod
  from typing import List, Tuple
  import torch
label_desc: Use image node_id 8 for calling the HybridEngineContainer obj's underlying member method code with example usage: obj.apply_tensor_parallelism(mp_replace, reversed_dim) without return types