How does backpropagation work with averaging layers?
Highly unlikely.

Simple thought experiment. If the vertices (graphically == weights) are the same for these two nodes (first cell of the vector for the first word and the first cell of the vector for the second word in context) (what I am trying to say they are fully connected) BUT input values when you feed-fowarded these values were different, not only that this difference spaned accros other layers, hence you can not expect that weight updates via backprop will be same.