How is get predict accuracy score in Bert Classification
I am using Bert Classifier for my Chatbot project. I perform the necessary tokenizer operations for the incoming text message. Then I insert it into the model and make a prediction. How can I get the accuracy of this estimate?
for text in test_texts:
    encoded_dict = tokenizer.encode_plus(
        text,
        add_special_tokens=True,
        max_length=max_len,
        pad_to_max_length=True,
        return_attention_mask=True,
        return_tensors='pt',
    )

    input_ids.append(encoded_dict['input_ids'])
    attention_masks.append(encoded_dict['attention_mask'])

input_ids = torch.cat(input_ids, dim=0)
attention_masks = torch.cat(attention_masks, dim=0)
print(&quot;input_ids &quot;,input_ids)
print(&quot;attention_masks &quot;,attention_masks)

batch_size = 32

prediction_data = TensorDataset(input_ids, attention_masks)
prediction_sampler = SequentialSampler(prediction_data)
prediction_dataloader = DataLoader(prediction_data, sampler=prediction_sampler, batch_size=batch_size)

print(&quot;prediction_data &quot;,prediction_data)
print(&quot;prediction_sampler &quot;,prediction_sampler)
print(&quot;prediction_dataloader &quot;,prediction_dataloader)
model.eval()
predictions, true_labels = [], []

for batch in prediction_dataloader:
    batch = tuple(t.to(device) for t in batch)
    b_input_ids, b_input_mask = batch
    print(&quot;b input ids&quot;,b_input_ids)
    with torch.no_grad():
        outputs = model(b_input_ids, token_type_ids=None,
                        attention_mask=b_input_mask.to(device))

    logits = outputs[0]
    logits = logits.detach().cpu().numpy()
    label_ids = b_input_mask.to('cpu').numpy()

    predictions.append(logits)
    true_labels.append(label_ids)
    print(&quot;logits &quot;,logits)

    print(&quot;label_ids &quot;,label_ids)
    print(&quot;true_labels &quot;,true_labels)

print('Prediction completed')

prediction_set = []

for i in range(len(true_labels)):
    pred_labels_i = np.argmax(predictions[i], axis=1).flatten()
    prediction_set.append(pred_labels_i)

prediction= [item for sublist in prediction_set for item in sublist]


print(&quot;prediction:&quot;, prediction[0])

I am looking for a percentage value. will respond or pass depending on the result of this percentage value.

Accuracy can be directly computed using some libraries.
For example, you can use sklearn:
from sklearn.metrics import accuracy_score
print(&quot;Accuracy:&quot;, accuracy_score(true_labels, predictions)) # Value between 0 and 1

print(&quot;Accuracy Percentage {} %:&quot;.format(100*accuracy_score(true_labels, predictions))) # Value between 0 and 100
``


