What is the position embedding code? https://github.com/google-research/bert/blob/master/modeling.py#L491-L520

The code of BERT is one of the implementation. But it is not what I need.

I search a lot but can not judge.

But where is the code which seems uses cos and sin?
Here is the code implemented by TensorFlow:

Implementation 1:




Implementation 2: