Bidirectional Encoder Representations from Transformers in R
You might be interested in the open-source R package RBERT:
https://github.com/jonathanbratt/RBERT

It's a work in progress, but the goal is to be able to use BERT directly in R.