Keras' Evaluate function training model on test set
Keras doesn't update your model with testing data. 
It might be that your labels have been provided wrong in the test data, check the model.predict_classes() to get the output of your classes and crosscheck them with your actual output manually by picking random subset.