How to use BERT as a multi-purpose conversational AI?
Think of BERT (or similar models) as as good starting place for understanding context.  

A couple options to make BERT contextualize dialogue:


Concatenate all messages with a seperator embedding and finetune a language model like BERT


This has shown good results in this paper, but understand it has weaknesses like struggling to determine order or author comprehension or conversation disentanglement. Additionally as threads grow longer this becomes more and more expensive  

Use A BERT like model as an RNN and pass along author/memory information as the hidden state


Weaknesses here is that you are putting a lot of strain on the hidden state of the model, and ideally you want to train each comment block separately so the gradient wouldnt need to unroll or traverse the network in backprop (I actually implemented a variant of this for my work, and had good results)



The 2 modes above are just baseline examples that you can alter for your own needs, and im sure there is also others if you brainstorm about it.

Note if you want generation capabilities, you would want to use GPT2 or XLNet instead of BERT for unidirectional embeddings. Hope this helped.