How to get sentence embedding using BERT? How to get sentence embedding using BERT?
from transformers import BertTokenizer
tokenizer=BertTokenizer.from_pretrained('bert-base-uncased')
sentence='I really enjoyed this movie a lot.'
#1.Tokenize the sequence:
tokens=tokenizer.tokenize(sentence)
print(tokens)
print(type(tokens))

2. Add [CLS] and [SEP] tokens:
tokens = ['[CLS]'] + tokens + ['[SEP]']
print(&quot; Tokens are \n {} &quot;.format(tokens))

3. Padding the input:
T=15
padded_tokens=tokens +['[PAD]' for _ in range(T-len(tokens))]
print(&quot;Padded tokens are \n {} &quot;.format(padded_tokens))
attn_mask=[ 1 if token != '[PAD]' else 0 for token in padded_tokens  ]
print(&quot;Attention Mask are \n {} &quot;.format(attn_mask))

4. Maintain a list of segment tokens:
seg_ids=[0 for _ in range(len(padded_tokens))]
print(&quot;Segment Tokens are \n {}&quot;.format(seg_ids))

5. Obtaining indices of the tokens in BERT’s vocabulary:
sent_ids=tokenizer.convert_tokens_to_ids(padded_tokens)
print(&quot;senetence idexes \n {} &quot;.format(sent_ids))
token_ids = torch.tensor(sent_ids).unsqueeze(0) 
attn_mask = torch.tensor(attn_mask).unsqueeze(0) 
seg_ids   = torch.tensor(seg_ids).unsqueeze(0)

Feed them to BERT
hidden_reps, cls_head = bert_model(token_ids, attention_mask = attn_mask,token_type_ids = seg_ids)
print(type(hidden_reps))
print(hidden_reps.shape ) #hidden states of each token in inout sequence 
print(cls_head.shape ) #hidden states of each [cls]

output:
hidden_reps size 
torch.Size([1, 15, 768])

cls_head size
torch.Size([1, 768])

Which vector represents the sentence embedding here? Is it hidden_reps  or cls_head ?
Is there any other way to get sentence embedding from BERT in order to perform similarity check with other sentences?
There is actually an academic paper for doing so. It is called S-BERT or Sentence-BERT.
They also have a github repo which is easy to work with.