class_docstr: This layer applies a self-attention with linear complexity, as described in `MobileViTv2 <https://arxiv.org/abs/2206.02680>`_ paper.
  This layer can be used for self- as well as cross-attention.
  
  Args:
      opts: command line arguments
      embed_dim (int): :math:`C` from an expected input of size :math:`(N, C, H, W)`
      attn_dropout (Optional[float]): Dropout value for context scores. Default: 0.0
      bias (Optional[bool]): Use bias in learnable layers. Default: True
  
  Shape:
      - Input: :math:`(N, C, P, N)` where :math:`N` is the batch size, :math:`C` is the input channels,
      :math:`P` is the number of pixels in the patch, and :math:`N` is the number of patches
      - Output: same as the input
  
  .. note::
      For MobileViTv2, we unfold the feature map [B, C, H, W] into [B, C, P, N] where P is the number of pixels
      in a patch and N is the number of patches. Because channel is the first dimension in this unfolded tensor,
      we use point-wise convolution (instead of a linear layer). This avoids a transpose operation (which may be
      expensive on resource-constrained devices) that may be required to convert the unfolded tensor from
      channel-first to channel-last format in case of a linear layer.
class_docstr_tok: ["This","layer","applies","a","self-attention","with","linear","complexity",",","as","described","in","`","MobileViTv2","<","https",":","\/\/arxiv.org\/abs\/2206.02680",">","`","_","paper",".","This","layer","can","be","used","for","self-","as","well","as","cross-attention",".","Args",":","opts",":","command","line","arguments","embed_dim","(","int",")",":",":","math",":","`","C","`","from","an","expected","input","of","size",":","math",":","`","(","N",",","C",",","H",",","W",")","`","attn_dropout","(","Optional","[","float","]",")",":","Dropout","value","for","context","scores",".","Default",":","0.0","bias","(","Optional","[","bool","]",")",":","Use","bias","in","learnable","layers",".","Default",":","True","Shape",":","-","Input",":",":","math",":","`","(","N",",","C",",","P",",","N",")","`","where",":","math",":","`","N","`","is","the","batch","size",",",":","math",":","`","C","`","is","the","input","channels",",",":","math",":","`","P","`","is","the","number","of","pixels","in","the","patch",",","and",":","math",":","`","N","`","is","the","number","of","patches","-","Output",":","same","as","the","input","..","note",":",":","For","MobileViTv2",",","we","unfold","the","feature","map","[","B",",","C",",","H",",","W","]","into","[","B",",","C",",","P",",","N","]","where","P","is","the","number","of","pixels","in","a","patch","and","N","is","the","number","of","patches",".","Because","channel","is","the","first","dimension","in","this","unfolded","tensor",",","we","use","point-wise","convolution","(","instead","of","a","linear","layer",")",".","This","avoids","a","transpose","operation","(","which","may","be","expensive","on","resource-constrained","devices",")","that","may","be","required","to","convert","the","unfolded","tensor","from","channel-first","to","channel-last","format","in","case","of","a","linear","layer","."]
code: def forward(
      self, x: Tensor, x_prev: Optional[Tensor] = None, *args, **kwargs
  ) -> Tensor:
      if x_prev is None:
          return self._forward_self_attn(x, *args, **kwargs)
      else:
          return self._forward_cross_attn(
              x, x_prev=x_prev, *args, **kwargs
          )
  
file_path: ml-cvnets/cvnets/layers/linear_attention.py
imports: from typing import Optional
  import torch
  from torch import Tensor
  from torch.nn import functional
  from cvnets.layers.base_layer import BaseLayer
  from cvnets.layers.conv_layer import ConvLayer2d
  from cvnets.layers.dropout import Dropout
label_desc: Use image node_id 6 for calling the LinearSelfAttention obj's underlying member method code with example usage: obj.forward(x, x_prev) and returns: self, self