def multiplicative_self_attention(units, n_hidden=None, n_output_features=None, activation=None):
n_input_features = K.int_shape(units)[<NUM_LIT:2>]<EOL>if n_hidden is None:<EOL><INDENT>n_hidden = n_input_features<EOL><DEDENT>if n_output_features is None:<EOL><INDENT>n_output_features = n_input_features<EOL><DEDENT>exp1 = Lambda(lambda x: expand_tile(x, axis=<NUM_LIT:1>))(units)<EOL>exp2 = Lambda(lambda x: expand_tile(x, axis=<NUM_LIT:2>))(units)<EOL>queries = Dense(n_hidden)(exp1)<EOL>keys = Dense(n_hidden)(exp2)<EOL>scores = Lambda(lambda x: K.sum(queries * x, axis=<NUM_LIT:3>, keepdims=True))(keys)<EOL>attention = Lambda(lambda x: softmax(x, axis=<NUM_LIT:2>))(scores)<EOL>mult = Multiply()([attention, exp1])<EOL>attended_units = Lambda(lambda x: K.sum(x, axis=<NUM_LIT:2>))(mult)<EOL>output = Dense(n_output_features, activation=activation)(attended_units)<EOL>return output<EOL>
Compute multiplicative self attention for time series of vectors (with batch dimension)
the formula: score(h_i, h_j) = <W_1 h_i,  W_2 h_j>,  W_1 and W_2 are learnable matrices
with dimensionality [n_hidden, n_input_features]

Args:
    units: tf tensor with dimensionality [batch_size, time_steps, n_input_features]
    n_hidden: number of units in hidden representation of similarity measure
    n_output_features: number of features in output dense layer
    activation: activation at the output

Returns:
    output: self attended tensor with dimensionality [batch_size, time_steps, n_output_features]