Trained Tensorflow model performs poorly on inference
I believe the samples that you are using for inference belongs to a different distribution of data from training and test set. So, I would suggest you to verify if the above case is true and if it is then you should fine-tune your model for the examples that you are trying to carry out inference.