class_docstr: PETransformerEncoderLayer is made up of Positional Encoding (PE), residual connections, self-attn and feedforward network.
  Major differences between this implementation and the pytorch official torch.nn.TransformerEncoderLayer are:
  1. Augment input data with positional encoding. hat{x} = x + PE{x}
  2. Two paralle residual blocks are applied to the raw input data (x) and encoded input data (hat{x}), respectively, i.e. z = Residual(x), hat{z} = Residual(hat{x})
  3. Treat z as the Value input, and hat{z} as the Query and Key input to feed a self-attention block.
  
  Main Args:
      d_model: the number of expected features in the input (required).
      nhead: the number of heads in the multiheadattention models (required).
      dim_feedforward: the dimension of the feedforward network model (default=2048).
      activation: the activation function of intermediate layer, relu or gelu (default=relu).
      layer_norm_eps: the eps value in layer normalization components (default=1e-5).
      batch_first: If ``True``, then the input and output tensors are provided
          as (batch, seq, feature). Default: ``False``.
      max_len: argument passed to the Positional Encoding module, see more details in the PositionalEncoding class.
class_docstr_tok: ["PETransformerEncoderLayer","is","made","up","of","Positional","Encoding","(","PE",")",",","residual","connections",",","self-attn","and","feedforward","network",".","Major","differences","between","this","implementation","and","the","pytorch","official","torch.nn.TransformerEncoderLayer","are",":","1",".","Augment","input","data","with","positional","encoding",".","hat","{","x","}","=","x","+","PE","{","x","}","2",".","Two","paralle","residual","blocks","are","applied","to","the","raw","input","data","(","x",")","and","encoded","input","data","(","hat","{","x","}",")",",","respectively",",","i.e",".","z","=","Residual","(","x",")",",","hat","{","z","}","=","Residual","(","hat","{","x","}",")","3",".","Treat","z","as","the","Value","input",",","and","hat","{","z","}","as","the","Query","and","Key","input","to","feed","a","self-attention","block",".","Main","Args",":","d_model",":","the","number","of","expected","features","in","the","input","(","required",")",".","nhead",":","the","number","of","heads","in","the","multiheadattention","models","(","required",")",".","dim_feedforward",":","the","dimension","of","the","feedforward","network","model","(","default=2048",")",".","activation",":","the","activation","function","of","intermediate","layer",",","relu","or","gelu","(","default=relu",")",".","layer_norm_eps",":","the","eps","value","in","layer","normalization","components","(","default=1e-5",")",".","batch_first",":","If","``","True","``",",","then","the","input","and","output","tensors","are","provided","as","(","batch",",","seq",",","feature",")",".","Default",":","``","False","``",".","max_len",":","argument","passed","to","the","Positional","Encoding","module",",","see","more","details","in","the","PositionalEncoding","class","."]
code: def __setstate__(self, state):
      if "activation" not in state:
          state["activation"] = F.relu
      super(PETransformerEncoderLayer, self).__setstate__(state)
  
imports: import logging
  import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
  from reagent.core import parameters, types
  from reagent.models import convolutional_network, fully_connected_network
  from reagent.models.base import ModelBase
  from reagent.models.fully_connected_network import ACTIVATION_MAP
label_desc: Use image node_id 2 for calling the PETransformerEncoderLayer obj's underlying member method code with example usage: obj.__setstate__(state) without return types