Similarity of words using BERTMODEL
First of all, I think you are confused with pretrained and finetuned.

BERT is pretrained on a lot of text data. By using this pretrained BERT, you have a model that already have knowledge about text.

BERT can then be finetuned on specific dataset, where BERT learn specific knowledge related to the dataset. That's why a finetuned BERT is bad on other datasets : the knowledge does not apply.

You have a custom dataset. So you don't want to use a finetuned model, but you want to still use a pretrained model (to have access to the general knowledge of BERT).

If you use a BERT model from scratch (not pretrained), you're basically creating an empty model, randomly initialized : it's useless.



Now that we have clarified this, let's see your problem.

What you are doing is calling the constructor of BertModel with the arguments of the forward pass. You first need to create the model, and then use the forward pass :

from transformers import BertModel, BertConfig
config = BertConfig(...)    # Here your parameters to initialize your BERT as you want, such as number of heads, etc...
model = BertModel(config)
hidden_reps, cls_head = model(token_ids, attention_mask=attn_mask, token_type_ids=seg_ids)




BUT, as I explained in the first part, you should not do this : it's completely useless and your results are going to be random. You want to use a pretrained BERT, to have some meaningful results :

from transformers import BertModel
model = BertModel.from_pretrained('bert-base-uncased')
hidden_reps, cls_head = model(token_ids, attention_mask=attn_mask, token_type_ids=seg_ids)


Note : Your results can be improved further if you finetune your BERT model on your custom dataset. 



Final point : Don't reinvent the wheel

People already tried to use BERT for word similarity. Instead of implementing this from scratch, using only a pretrained model, potentially adding bug to your own implementation, just use some already existing code !

I'm not sure about NER, but for word similarity, you can take a look at BERT Score. Their model is already finetuned for word similarity, so it's likely that you will get higher score using this approach, and also much easier path to take.