Should I choose a model with the smallest loss or highest accuracy?
You should choose the model A. The loss is just a differentiable proxy for accuracy.

That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine the data which produce high loss and check for presence of overfitting or incorrect labels. 

If the higher loss is due to a regularizer then reducing the regularization factor may further improve the results.