Deep learning model performance for different datasets?
I can think of one reason. Because you're using different data sets, the properties of your data set are likely to be different. So obviously what the model will learn will be different. And maybe the architecture of your network suits one data set more than the other. But also if you try training the model on your other data set(with less accuracy) for more epochs, it's possible that the accuracy will improve. Your model could be just be slow to understand this data set.

It's like when you see a human you can easily recognize a face. But when you see a software code, it's not always easy to find the bug. But if you're trained well enough you'll eventually be faster and better in recognizing the bugs.

I hope it makes sense to you.