class_docstr: A middleware embedder is the middle part of the network. It takes the embeddings from the input embedders,
  after they were aggregated in some method (for example, concatenation) and passes it through a neural network
  which can be customizable but shared between the heads of the network
code: def __init__(
      self,
      activation_function=tf.nn.relu,
      scheme: MiddlewareScheme = MiddlewareScheme.Medium,
      batchnorm: bool = False,
      dropout_rate: float = 0.0,
      name="middleware_embedder",
      dense_layer=Dense,
      is_training=False,
  ):
      self.name = name
      self.input = None
      self.output = None
      self.activation_function = activation_function
      self.batchnorm = batchnorm
      self.dropout_rate = dropout_rate
      self.scheme = scheme
      self.return_type = MiddlewareEmbedding
      self.dense_layer = dense_layer
      if self.dense_layer is None:
          self.dense_layer = Dense
      self.is_training = is_training
  
      # layers order is conv -> batchnorm -> activation -> dropout
      if isinstance(self.scheme, MiddlewareScheme):
          self.layers_params = copy.copy(self.schemes[self.scheme])
          self.layers_params = [
              convert_layer(l) for l in self.layers_params
          ]
      else:
          # if scheme is specified directly, convert to TF layer if it's not a callable object
          # NOTE: if layer object is callable, it must return a TF tensor when invoked
          self.layers_params = [
              convert_layer(l) for l in copy.copy(self.scheme)
          ]
  
      # we allow adding batchnorm, dropout or activation functions after each layer.
      # The motivation is to simplify the transition between a network with batchnorm and a network without
      # batchnorm to a single flag (the same applies to activation function and dropout)
      if (
          self.batchnorm
          or self.activation_function
          or self.dropout_rate > 0
      ):
          for layer_idx in reversed(range(len(self.layers_params))):
              self.layers_params.insert(
                  layer_idx + 1,
                  BatchnormActivationDropout(
                      batchnorm=self.batchnorm,
                      activation_function=self.activation_function,
                      dropout_rate=self.dropout_rate,
                  ),
              )
  
file_path: coach/rl_coach/architectures/tensorflow_components/middlewares/middleware.py
imports: import copy
  from typing import Union, Tuple
  import tensorflow
  from rl_coach.architectures.tensorflow_components.layers import BatchnormActivationDropout, convert_layer, Dense
  from rl_coach.base_parameters import MiddlewareScheme, NetworkComponentParameters
  from rl_coach.core_types import MiddlewareEmbedding
label_desc: Use image node_id 1 to create a new Middleware object from inherited base classes: object with example: obj = Middleware(activation_function, scheme, batchnorm, dropout_rate, name, dense_layer, is_training)