class_docstr: This class defines the pre-norm `Transformer encoder <https://arxiv.org/abs/1706.03762>`_
  with the addition of windowed attention.
  
  This class first partitions the input sequence into a series of windows (with
  an optional offset to use when defining windows). Then, it calls a
  TransformerEncoder module. Then, it undoes windowing.
  
  Args:
      opts: Command line arguments.
      embed_dim: :math:`C_{in}` from an expected input of size :math:`(N, P, C_{in})`.
      ffn_latent_dim: Inner dimension of the FFN.
      num_heads: Number of heads in multi-head attention. Default: 8.
      attn_dropout: Dropout rate for attention in multi-head attention. Default: 0.0.
      dropout: Dropout rate. Default: 0.0.
      ffn_dropout: Dropout between FFN layers. Default: 0.0.
      transformer_norm_layer: Normalization layer. Default: layer_norm.
      stochastic_dropout: Stochastic dropout setting. Default: 0.0.
      window_size: The size of the window, if using windowed attention. Default: None.
      window_shift: The size of the shift, if using shifted windowed attention. Default: None.
  
  Shape:
      - Input: :math:`(N, P, C_{in})` where :math:`N` is batch size, :math:`P` is number of patches,
      and :math:`C_{in}` is input embedding dim
      - Output: same shape as the input
code: def __init__(
      self,
      opts: argparse.Namespace,
      embed_dim: int,
      ffn_latent_dim: int,
      num_heads: Optional[int] = 8,
      attn_dropout: Optional[float] = 0.0,
      dropout: Optional[float] = 0.0,
      ffn_dropout: Optional[float] = 0.0,
      transformer_norm_layer: Optional[str] = "layer_norm",
      stochastic_dropout: Optional[float] = 0.0,
      window_size: Optional[int] = None,
      window_shift: Optional[int] = None,
      *args,
      **kwargs,
  ) -> None:
      super().__init__(
          opts=opts,
          embed_dim=embed_dim,
          ffn_latent_dim=ffn_latent_dim,
          num_heads=num_heads,
          attn_dropout=attn_dropout,
          dropout=dropout,
          ffn_dropout=ffn_dropout,
          transformer_norm_layer=transformer_norm_layer,
          stochastic_dropout=stochastic_dropout,
      )
      if window_size is None:
          raise ValueError("Please specify window_size")
      if window_shift is None:
          raise ValueError("Please specify window_shift")
      self.window_size: int = window_size
      self.window_shift: int = window_shift
  
file_path: ml-cvnets/cvnets/modules/windowed_transformer.py
imports: import argparse
  from typing import Optional, Tuple
  import torch
  from torch import Tensor
  from cvnets.layers import token_merging
  from cvnets.modules import transformer
label_desc: Use image node_id 1 to create a new WindowedTransformerEncoder object from inherited base classes: transformer with example: obj = WindowedTransformerEncoder(opts, embed_dim, ffn_latent_dim, num_heads, attn_dropout, dropout, ffn_dropout, transformer_norm_layer, stochastic_dropout, window_size, window_shift)