Maximum Allowed Time for Short Circuit
If thermal heating effect of the short circuit on a device is considered, the short should be removed before the device temperature rises higher than allowed. The repetition of such events will increase the probability of failure of the device 
If the short circuit (over current protection) is implemented as a safety mechanism, it depends on the system being protected. If the device is sensitive or the protection of the user is of utmost priority (safety) then there are fast acting fuses 
Slow fuses or PTCs also work as current limiting but slower. Here, it is assumed that the risk of the device failing permanently will be less. 
If I am using a small fuse for a motor driver circuit,I don't want the fuse to trip every time motor takes the surge current. But when the motor is stalled, it shouldn't be in that condition for longer. Here I can use slow resettable fuses..for example.  
Rule of thumb is application dependant. A shock for a person even for 20 ms can make his heart beat miss its rhythm.  
Little fuse document as reference: defining the application type
 
The PTCs are limited by number of such limit events, they are rated for such number of cycles.