Tensorflow: how to look up and average a different amount of embedding vectors per training instance, with multiple training instances per minibatch?
Use tf.gather(). 

Single instance case

In the example below, we selected a variable number of embedding vectors from the matrix embedding. The selection indexing vector user can be of variable length. Then we calculate the average embedding.



with tf.Graph().as_default():
    embedding = tf.placeholder(shape=[10,3], dtype=tf.float32)
    user = tf.placeholder(shape=None, dtype=tf.int32)
    selected = tf.gather(embedding, user)
    average = tf.reduce_mean(selected, axis=0)

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        embedding_ = np.random.randn(10,3)
        user_ = [1,3,5]
        print(sess.run(average, feed_dict={embedding:embedding_, user:user_}))
        print(np.mean(embedding_[user_], axis=0))


Multiple instances in a mini-batch

You may manually specify the first vector in embedding to be a zero vector, and patch the above selection vector with 0s. For example 


with tf.Graph().as_default():
    embedding = tf.placeholder(shape=[10,3], dtype=tf.float32)
    user = tf.placeholder(shape=[None, None], dtype=tf.int32)
    selected = tf.gather(embedding, user)
    non_zero_count =  tf.cast(tf.count_nonzero(user, axis=1), tf.float32)
    embedding_sum = tf.reduce_sum(selected, axis=1) 
    average = embedding_sum / tf.expand_dims(non_zero_count, axis=1)

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        embedding_ = np.concatenate([np.zeros((1,3)),np.random.randn(9,3)], axis=0)
        user_ = [[3,5,7,0], [1,2,0,0]]
        print(sess.run(average, feed_dict={embedding:embedding_, user:user_}))
        print(np.sum([embedding_[i] for i in user_], axis=1) / np.atleast_2d(np.count_nonzero(user_, axis=1)).T)


You can use tf.gather() like this even if the embedding is a trainable variable instead of a placeholder.