class_docstr: Computes a weighted average of the different channels across timesteps.
  Uses 1 parameter pr. channel to compute the attention value for
  a single timestep.
class_docstr_tok: ["Computes","a","weighted","average","of","the","different","channels","across","timesteps",".","Uses","1","parameter","pr",".","channel","to","compute","the","attention","value","for","a","single","timestep","."]
code: def call(self, x, mask=None):
      # computes a probability distribution over the timesteps
      # uses 'max trick' for numerical stability
      # reshape is done to avoid issue with Tensorflow
      # and 1-dimensional weights
      logits = K.dot(x, self.W)
      x_shape = K.shape(x)
      logits = K.reshape(logits, (x_shape[0], x_shape[1]))
      ai = K.exp(logits - K.max(logits, axis=-1, keepdims=True))
  
      # masked timesteps have zero weight
      if mask is not None:
          mask = K.cast(mask, K.floatx())
          ai = ai * mask
      att_weights = ai / (
          K.sum(ai, axis=1, keepdims=True) + K.epsilon()
      )
      weighted_input = x * K.expand_dims(att_weights)
      result = K.sum(weighted_input, axis=1)
      if self.return_attention:
          return [result, att_weights]
      return result
  
file_path: textgenrnn/textgenrnn/AttentionWeightedAverage.py
imports: from tensorflow.keras.layers import Layer, InputSpec
  from tensorflow.keras import backend
  from tensorflow.keras import initializers
label_desc: Use image node_id 3 for calling the AttentionWeightedAverage obj's underlying member method code with example usage: obj.call(x, mask) and returns: result, list