object image segmentation
That's a tough one.  I don't know what could cause that.
It can be hard to know why we see the results we do, when working with neural nets.  Often the best we can do is form several hypotheses, and then devise experiments to try to test those hypotheses.
Some possible explanations that you could try to test:

Perhaps it is noise / random chance

Perhaps the 50-pixel images are fairly homogenous and the 150-pixel images are fairly homogenous, and taking a combination of these two yields more diversity, which helps the network learn (e.g., it doesn't overfit to either)

Perhaps the 50-pixel objects don't come from the same distribution as the 150-pixel objects (e.g., the 50-pixel images are of children and the 150-pixel images are of adults; these differ in more ways than just size; or the background is correlated to the size of the object), and you get more regularization from exposing the network to both, or something (this is vague and might not be very different from the second explanation)


Maybe you can think of other possibilities.