How to add a CNN layer on top of BERT? I am just playing with bert (Bidirectional Encoder Representation from Transformer) 
Research Paper

Suppose I want to add any other model or layers like Convolutional Neural Network layers (CNN), Non Linear (NL) layers on top of BERT model. How can I do this?

I am not able to figure out where should I change in code of BERT. I am using the pytorch implementation of bert
from huggingface.

This is what I want to do:



Please show steps to implement this using sudo code which will help me in implemention of cnn on top of BERT.
You could use HuggingFace's BertModel (transformers) as the base layer for your model and just like how you would build a neural network in Pytorch, you can build on top of it. HuggingFace's other BertModels are built in the same way. For reference you can take a look at their TokenClassification code over here. This hasn't been mentioned in the documentation much and I think it should.

For Tensorflow however, you would have convert the Bert Model into a Keras layer. I haven't really gone through it much or tried it, but I think this blog post does a pretty good job of explaining it.