Why does Pytorch EmbeddingBag with mode "max" not accept `per_sample_weights`?
Pytorch's EmbeddingBag allows for efficient lookup + reduce operations on varying length collections of embedding indices. There are 3 modes: &quot;sum&quot;, &quot;average&quot; and &quot;max&quot; for the reduce operation. With &quot;sum&quot;, you can also provide per_sample_weights giving you a weighted sum.
Why is per_sample_weights not allowed for the &quot;max&quot; operation? Looking at how it's implemented, I can only assume there is an issue with performing a &quot;ReduceMean&quot; or &quot;ReduceMax&quot; operation after a &quot;Mul&quot; operation. Could that be something to do with calculating gradients??

p.s: It's easy enough to turn a weighted sum into a weighted average by dividing by the sum of the weights, but for &quot;max&quot; you can't get a weighted equivalent like that.

The argument per_sample_weights was only implemented for mode='sum', not due to technical limitations, but because the developers found no use cases for a &quot;weighted max&quot;:

I haven't been able to find use cases for &quot;weighted mean&quot; (which can be emulated via weighted sum) and &quot;weighted max&quot;.


Feature request: Weighted average for EmbeddingBag

