def multiplicative_self_attention(units, n_hidden=None, n_output_features=None, activation=None):
n_input_features = units.get_shape().as_list()[<NUM_LIT:2>]<EOL>if n_hidden is None:<EOL><INDENT>n_hidden = n_input_features<EOL><DEDENT>if n_output_features is None:<EOL><INDENT>n_output_features = n_input_features<EOL><DEDENT>queries = tf.layers.dense(expand_tile(units, <NUM_LIT:1>), n_hidden, kernel_initializer=INITIALIZER())<EOL>keys = tf.layers.dense(expand_tile(units, <NUM_LIT:2>), n_hidden, kernel_initializer=INITIALIZER())<EOL>scores = tf.reduce_sum(queries * keys, axis=<NUM_LIT:3>, keep_dims=True)<EOL>attention = tf.nn.softmax(scores, dim=<NUM_LIT:2>)<EOL>attended_units = tf.reduce_sum(attention * expand_tile(units, <NUM_LIT:1>), axis=<NUM_LIT:2>)<EOL>output = tf.layers.dense(attended_units, n_output_features, activation, kernel_initializer=INITIALIZER())<EOL>return output<EOL>
Computes multiplicative self attention for time series of vectors (with batch dimension)
        the formula: score(h_i, h_j) = <W_1 h_i,  W_2 h_j>,  W_1 and W_2 are learnable matrices
        with dimensionality [n_hidden, n_input_features], where <a, b> stands for a and b
        dot product

    Args:
        units: tf tensor with dimensionality [batch_size, time_steps, n_input_features]
        n_hidden: number of units in hidden representation of similarity measure
        n_output_features: number of features in output dense layer
        activation: activation at the output

    Returns:
        output: self attended tensor with dimensionality [batch_size, time_steps, n_output_features]