Why is there a time limit on Tobi's dematerialization technique?
There's no definite answer yet.

The technique is about sending parts of yourself to the other dimension, to avoid being hit. 

Two reasons I can think of are: 


Since it's an activated mode type technique, it constantly drains his chakra. Five minutes are his limit. 
He cannot keep parts of his body separated for so long, because there could be some sort of biological limitation of transferring oxygen and blood to the organs in the alternate dimension. 


The first sounds more feasible to me, but since it isn't explicitly specified, we can't give you a definite answer.