class_docstr: BLIP captioning model.
  
  Supported model types:
      - base_coco: fine-tuned BLIP base model on COCO caption dataset (Karparthy split).
      - large_coco: fine-tuned BLIP large model on COCO caption dataset (Karparthy split).
  
  Usage:
      >>> from lavis.models import load_model
      >>> model = load_model("blip_caption", "base_coco")
      >>> model = load_model("blip_caption", "large_coco")
class_docstr_tok: ["BLIP","captioning","model",".","Supported","model","types",":","-","base_coco",":","fine-tuned","BLIP","base","model","on","COCO","caption","dataset","(","Karparthy","split",")",".","-","large_coco",":","fine-tuned","BLIP","large","model","on","COCO","caption","dataset","(","Karparthy","split",")",".","Usage",":",">",">",">","from","lavis.models","import","load_model",">",">",">","model","=","load_model","(","``","blip_caption","''",",","``","base_coco","''",")",">",">",">","model","=","load_model","(","``","blip_caption","''",",","``","large_coco","''",")"]
docstr: Args:
      samples (dict): A dictionary containing the following keys:
          - image (torch.Tensor): A tensor of shape (batch_size, 3, H, W)
          - text_input (list): A list of strings of length batch_size.
  Returns:
      output (BlipOutput): A BlipOutput object containing the following
          attributes:
          - loss (torch.Tensor): A scalar tensor containing the total loss. For BlipCaption, this is the same as the LM loss.
          - loss_lm (torch.Tensor): A scalar tensor containing the LM loss.
          - intermediate_outputs (BlipIntermediateOutput): A BlipIntermediateOutput object containing intermediate outputs.
            see :class:`lavis.models.blip_models.blip_outputs.BlipOutput` for more details.
  
  Example:
  ```python
  >>> from PIL import Image
  >>> from lavis.models import load_model_and_preprocess
  >>> model, vis_processors, txt_processors = load_model_and_preprocess("blip_caption")
  >>> raw_image = Image.open("docs/data/merlion.png").convert("RGB")
  >>> image = vis_processors["eval"](raw_image).unsqueeze(0)
  >>> text_input = ["a large statue of a person spraying water from a fountain"]
  >>> samples = {"image": image, "text_input": text_input}
  >>> output = model(samples)
  >>> output.keys()
  odict_keys(['intermediate_output', 'loss', 'loss_lm'])
  >>> output.intermediate_output.image_embeds.shape
  torch.Size([1, 577, 768])
  >>> output.intermediate_output.decoder_labels.shape
  torch.Size([1, 13])
  ```
docstr_tok: ["Args",":","samples","(","dict",")",":","A","dictionary","containing","the","following","keys",":","-","image","(","torch.Tensor",")",":","A","tensor","of","shape","(","batch_size",",","3",",","H",",","W",")","-","text_input","(","list",")",":","A","list","of","strings","of","length","batch_size",".","Returns",":","output","(","BlipOutput",")",":","A","BlipOutput","object","containing","the","following","attributes",":","-","loss","(","torch.Tensor",")",":","A","scalar","tensor","containing","the","total","loss",".","For","BlipCaption",",","this","is","the","same","as","the","LM","loss",".","-","loss_lm","(","torch.Tensor",")",":","A","scalar","tensor","containing","the","LM","loss",".","-","intermediate_outputs","(","BlipIntermediateOutput",")",":","A","BlipIntermediateOutput","object","containing","intermediate","outputs",".","see",":","class",":","`","lavis.models.blip_models.blip_outputs.BlipOutput","`","for","more","details",".","Example",":","``","`","python",">",">",">","from","PIL","import","Image",">",">",">","from","lavis.models","import","load_model_and_preprocess",">",">",">","model",",","vis_processors",",","txt_processors","=","load_model_and_preprocess","(","``","blip_caption","''",")",">",">",">","raw_image","=","Image.open","(","``","docs\/data\/merlion.png","''",")",".convert","(","``","RGB","''",")",">",">",">","image","=","vis_processors","[","``","eval","''","]","(","raw_image",")",".unsqueeze","(","0",")",">",">",">","text_input","=","[","``","a","large","statue","of","a","person","spraying","water","from","a","fountain","''","]",">",">",">","samples","=","{","``","image","''",":","image",",","``","text_input","''",":","text_input","}",">",">",">","output","=","model","(","samples",")",">",">",">","output.keys","(",")","odict_keys","(","[","'intermediate_output","'",",","'loss","'",",","'loss_lm","'","]",")",">",">",">","output.intermediate_output.image_embeds.shape","torch.Size","(","[","1",",","577",",","768","]",")",">",">",">","output.intermediate_output.decoder_labels.shape","torch.Size","(","[","1",",","13","]",")","``","`"]
code: def forward(self, samples):
      r"""
      Args:
          samples (dict): A dictionary containing the following keys:
              - image (torch.Tensor): A tensor of shape (batch_size, 3, H, W)
              - text_input (list): A list of strings of length batch_size.
      Returns:
          output (BlipOutput): A BlipOutput object containing the following
              attributes:
              - loss (torch.Tensor): A scalar tensor containing the total loss. For BlipCaption, this is the same as the LM loss.
              - loss_lm (torch.Tensor): A scalar tensor containing the LM loss.
              - intermediate_outputs (BlipIntermediateOutput): A BlipIntermediateOutput object containing intermediate outputs.
                see :class:`lavis.models.blip_models.blip_outputs.BlipOutput` for more details.
  
      Example:
      ```python
      >>> from PIL import Image
      >>> from lavis.models import load_model_and_preprocess
      >>> model, vis_processors, txt_processors = load_model_and_preprocess("blip_caption")
      >>> raw_image = Image.open("docs/data/merlion.png").convert("RGB")
      >>> image = vis_processors["eval"](raw_image).unsqueeze(0)
      >>> text_input = ["a large statue of a person spraying water from a fountain"]
      >>> samples = {"image": image, "text_input": text_input}
      >>> output = model(samples)
      >>> output.keys()
      odict_keys(['intermediate_output', 'loss', 'loss_lm'])
      >>> output.intermediate_output.image_embeds.shape
      torch.Size([1, 577, 768])
      >>> output.intermediate_output.decoder_labels.shape
      torch.Size([1, 13])
      ```"""
  
      image_embeds = self.forward_encoder(samples)
      decoder_output, decoder_targets = self.forward_decoder(
          samples, image_embeds
      )
  
      # return decoder_out
      return BlipOutput(
          loss=decoder_output.loss,
          loss_lm=decoder_output.loss,
          intermediate_output=BlipIntermediateOutput(
              image_embeds=image_embeds,
              decoder_output=decoder_output,
              decoder_labels=decoder_targets,
          ),
      )
  
file_path: lavis/lavis/models/blip_models/blip_caption.py
imports: import torch
  from lavis.common.registry import registry
  from lavis.models.blip_models.blip import BlipBase
  from lavis.models.blip_models.blip_outputs import BlipOutput, BlipIntermediateOutput
  from lavis.models.med import XBertLMHeadDecoder
  from lavis.models.vit import VisionTransformerEncoder
label_desc: Use image node_id 4 for calling the BlipCaption obj's underlying member method code with example usage: obj.forward(samples) and returns: BlipOutput