class_docstr: A retrival wrapper using mm_encoder as both video/text backbone.
  TODO: move formally.
code: def forward_text(
      self, caps, cmasks, output_hidden_states=False, **kwargs
  ):
      input_ids = torch.cat(
          [
              caps[:, :1],
              caps[:, 2:],
          ],
          dim=1,
      )
  
      attention_mask = torch.cat([cmasks[:, :1], cmasks[:, 2:]], dim=1)
  
      token_type_ids = torch.cat(
          [
              torch.zeros(
                  (cmasks.size(0), 1),
                  dtype=torch.long,
                  device=cmasks.device,
              ),
              torch.ones(
                  (cmasks.size(0), cmasks.size(1) - 2),
                  dtype=torch.long,
                  device=cmasks.device,
              ),
          ],
          dim=1,
      )
  
      outputs = self.mm_encoder(
          input_ids=input_ids,
          input_video_embeds=None,
          attention_mask=attention_mask,
          token_type_ids=token_type_ids,
          output_hidden_states=True,
      )
      text_outputs = outputs[0]
  
      if output_hidden_states:
          return text_outputs
  
      batch_size = caps.size(0)
      # text tokens + [SEP]
      text_attention_mask = torch.cat(
          [
              torch.zeros(
                  (batch_size, 1),
                  dtype=torch.bool,
                  device=cmasks.device,
              ),
              cmasks[:, 2:],
          ],
          dim=1,
      )
  
      assert text_outputs.size(1) == text_attention_mask.size(1)
  
      text_attention_mask = text_attention_mask.type(
          text_outputs.dtype
      ) / text_attention_mask.sum(1, keepdim=True)
  
      pooled_text = torch.bmm(
          text_outputs.transpose(2, 1), text_attention_mask.unsqueeze(2)
      ).squeeze(-1)
      return pooled_text
  
imports: import torch
  from torch import nn
  from .None import transformermodel
label_desc: Use image node_id 3 for calling the MMFusionShare obj's underlying member method code with example usage: obj.forward_text(caps, cmasks, output_hidden_states) and returns: pooled_text, text_outputs