class_docstr: A feature extractor consisting of a given CNN backbone followed by a standard
  Transformer-Encoder. See documentation of "ContextCNNGeneric" for more
  information.
  
  Args:
      cnn (nn.Module): any convolutional neural network that takes in batches of images of
           shape (batch_size, channels, height, width) and outputs tensor
           representations of shape (batch_size, out_channels, out_height, out_width) (required).
      cnn_output_shape (tuple): a tuple of shape (batch_size, num_channels, height, width)
                         describing the output shape of the given CNN (required).
      num_layers (int): number of attention layers in the Transformer-Encoder (required).
      num_heads (int): number of attention heads in each transformer block (required).
      dim_feedforward (int): number of neurons in the intermediate dense layer of
                        each transformer feedforward block (required).
      dropout (float): dropout rate of the transformer layers (required).
      output_type (string): one of 'sequence' or 'spatial'. If Spatial then the final
                    output of the model, which is the sequence output of the
                    Transformer-Encoder, will be reshaped to resemble the
                    image-batch shape of the output of the CNN (required).
      positional_encoder (nn.Module): None or a nn.Module that expects inputs of
                          shape (sequence_length, batch_size, embedding_dim)
                          and returns the same input after adding
                          some positional information to the embeddings. If
                          `None`, then the default and fixed sin-cos positional
                          encodings of base transformers are applied (optional).
  
  Examples:
      See pykale/examples/cifar_cnntransformer/model.py
code: def __init__(
      self,
      cnn: nn.Module,
      cnn_output_shape: Tuple[int, int, int, int],
      num_layers: int,
      num_heads: int,
      dim_feedforward: int,
      dropout: float,
      output_type: str,
      positional_encoder: nn.Module = None,
  ):
      num_channels = cnn_output_shape[1]
      height = cnn_output_shape[2]
      width = cnn_output_shape[3]
  
      encoder_layer = nn.TransformerEncoderLayer(
          num_channels, num_heads, dim_feedforward, dropout
      )
      encoder_normalizer = nn.LayerNorm(num_channels)
      encoder = nn.TransformerEncoder(
          encoder_layer, num_layers, encoder_normalizer
      )
  
      if positional_encoder is None:
          positional_encoder = PositionalEncoding(
              d_model=num_channels, max_len=height * width
          )
      else:
          # allows for passing the identity block to skip this step
          # or chosing a different encoding
          positional_encoder = positional_encoder
  
      transformer_input_dropout = nn.Dropout(dropout)
      contextualizer = nn.Sequential(
          positional_encoder, transformer_input_dropout, encoder
      )
  
      super(CNNTransformer, self).__init__(
          cnn, cnn_output_shape, contextualizer, output_type
      )
  
      # Copied from https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html#Transformer
      for p in encoder.parameters():
          if p.dim() > 1:
              nn.init.xavier_uniform_(p)
  
file_path: pykale/kale/embed/attention_cnn.py
imports: from typing import Any, Tuple, Union
  import torch
  import torch.nn
  from kale.embed.positional_encoding import PositionalEncoding
  from kale.prepdata.tensor_reshape import seq_to_spatial, spatial_to_seq
label_desc: Use image node_id 1 to create a new CNNTransformer object from inherited base classes: ContextCNNGeneric with example: obj = CNNTransformer(cnn, cnn_output_shape, num_layers, num_heads, dim_feedforward, dropout, output_type, positional_encoder)