Trying to implement a "smart compose" feature I found this post on Gmail's smart compose feature, and it got me thinking about trying to implement it myself.

https://ai.googleblog.com/2018/05/smart-compose-using-neural-networks-to.html

The text is super vague though - the only part that gives any direction at all is "we combined a BoW model with an RNN-LM".

I remember that BoW is an unordered representation of the input corpus where only relative magnitudes are important. But I just don't have enough experience to know how an unordered representation like this could help me predict an ordered suffix.

Does anybody have any thoughts on how this might more specifically work? Or any references for further reading? I wasn't able to find much unfortunately.
From the same blog,


  In this hybrid approach, we encode the subject and previous email by averaging the word embeddings in each field. We then join those averaged embeddings, and feed them to the target sequence RNN-LM at every decoding step.


The BoW part of their hybrid approach is to get the general context of the email conversation by averaging the word embeddings in the subject and the previous email. Passing this vectorized representation of the general context at each step of the RNN-LM, which takes into account word order, helps get predictions that are more tailored to the subject of the conversation. 

Note that at each step, the RNN-LM is getting 3 types of input - 


Embedding for the previous word - For immediate context.
State vector - For localized context 
Subject and previous email averaged embeddings - For a wider, conversation-level context


Hopefully, it is clearer now, that the model incorporates order with the use of RNN-LM and the BoW part is to condition the output to the subject of the conversation.