Is transfer learning applied on only similar datasets only?
Let me start by making an acknowledgement that in machine learning intuitions can be wrong and it is not possible to know beforehand what would work and what wouldn't work. That's why machine learning is very much an experiment based method.
Having made this acknowledgement let me tell you my intuition. If you transfer learning from image processing problem to a different image processing problem then it shouldn't hurt if your network is deep enough. Most likely, a convolutional network's earlier layers find simple features that are useful to describe any image. Like horizontal lines and diagonal lines. Then, the deeper layers vary according to specificity of the dataset to find more complex features.
Even if you are working on a wildly different image problem, what you can try to do is keep the earlier layers constant during the learning and just learn the deeper layers. After some learning you can allow the earlier layers to change as well to do some fine tuning.