Can a textile fiber cause an electrical short?
While some polymers can be somewhat conductive (that's how some organic devices work), those fibers you mentioned cannot be considered conductive: sweat, for instance, is orders of magnitude more conductive. 

Instead, the problem is that those fibers are actually insulators, and therefore they will prevent a good contact between the two connectors. The contact surface will decrease and thus the contact resistance will increase. If such contacts are supposed to carry a large current (i.e. the power contacts), such increased resistance might induce eccessive voltage drop or even local overheating.