class_docstr: ## Compressive Transformer Layer
  
  This is the implementation of a single compressive transformer layer
docstr: * `d_model` is the token embedding size
  * `self_attn` is the [self attention module](../xl/relative_mha.html)
  * `feed_forward` is the [feed forward module](../feed_forward.html)
  * `dropout_prob` is the probability of dropping out after self attention and FFN
  * `compress` is the compression function $f_c$
docstr_tok: ["*","`","d_model","`","is","the","token","embedding","size","*","`","self_attn","`","is","the","[","self","attention","module","]","(","..","\/xl\/relative_mha.html",")","*","`","feed_forward","`","is","the","[","feed","forward","module","]","(","..","\/feed_forward.html",")","*","`","dropout_prob","`","is","the","probability","of","dropping","out","after","self","attention","and","FFN","*","`","compress","`","is","the","compression","function","$","f_c","$"]
code: def __init__(
      self,
      *,
      d_model: int,
      self_attn: RelativeMultiHeadAttention,
      feed_forward: FeedForward,
      dropout_prob: float,
      compress: Conv1dCompression,
  ):
      """
      * `d_model` is the token embedding size
      * `self_attn` is the [self attention module](../xl/relative_mha.html)
      * `feed_forward` is the [feed forward module](../feed_forward.html)
      * `dropout_prob` is the probability of dropping out after self attention and FFN
      * `compress` is the compression function $f_c$
      """
      super().__init__()
      self.compress = compress
      self.size = d_model
      self.self_attn = self_attn
      self.feed_forward = feed_forward
      self.dropout = nn.Dropout(dropout_prob)
      self.norm_self_attn = nn.LayerNorm([d_model])
      self.norm_ff = nn.LayerNorm([d_model])
  
imports: from typing import Optional, List
  import torch
  import torch.nn.functional
  from torch import nn
  from labml_helpers.module import Module, TypedModuleList
  from labml_nn.transformers.feed_forward import FeedForward
  from labml_nn.transformers.mha import PrepareForMultiHeadAttention
  from labml_nn.transformers.xl.relative_mha import RelativeMultiHeadAttention
  from labml_nn.utils import clone_module_list
label_desc: Use image node_id 1 to create a new CompressiveTransformerLayer object from inherited base classes: Module with example: obj = CompressiveTransformerLayer()