How can I tokenize a text file with BERT or something similar? I want to use the twitter datasets in a project and the tweet contents look something like this:

tweet_ID         tweet_text

12324124         some text here that has been twitted bla bla bla
35325323         some other text, trump, usa , merica ,etc.
56743563         bla bla text whatever tweet bla bla


Now I would like to end-up with a file that contains tweet_IDs and some vector encodings. I was reading about BERT, ROBERTA, etc. Is there a way to simply generate these encodings without writing a huge amount of boilerplate code?
With Huginface's Transformers, it should be doing with not much boilerplate.

A minimum example using PyTorch:

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute")).unsqueeze(0)  # Batch size 1
outputs = model(input_ids)
last_hidden_states = outputs[0]  # The last hidden-state is the first element of the output tuple


Depending on how data you want to process, you might want to speed it up a little bit. In that case, you should do it batches which would require doing padding and passing the padding mask to the model.