Why is the solid product washed with ice-cold solvent during in the last stages of recrystallisation?
Possible sources of impurities in the recrystallization process can be described in terms of phenomena occuring during coprecipitation and which are the basis of gravimetry:


inclusion (impurity occupies positions in the crystal lattice);
occlusion (impurity is physically captured within the crystal as it has been growing);
adsorption (impurity is adsorbed on the surface layer of the crystal);
postprecipitation (impurity forms secondary solid phase once recrystallization of the target stopped/slowed down).


Washing crystals with a fluid only addresses the last two phenomena, adsorbance and postprecipitation. Postprecipitation is rarely a problem as it occurs on a large time scale or when a really poor solvent has been chosen.

Properties of the washing liquid vastly depend on the properties of the compound being recrystallized. In the simplest case it's the same solvent (as in your example), but it can also be an electrolyte, hydrolysis suppressor or another solvent (e.g. distilled water) altogether.

As for the temperature of the washing liquid, it's actually better to use many small portions of the hot solvent as filtration of the hot solutions happens much faster, surface adsorption decreases with the temperature and removal of mother solution from the surface is faster. However, if solubility of the recrystallized product is significantly increasing with the temperature (and this is typically the case), then one has to sacrifice the efficiency of the washing process for the better yield and use cold solvent instead.

Impurities, insoluble in the cold solvent, are practically not an issue. First, recrystallization starts from a homogeneous solution. Second, before crystals are washed, they are separated from the mother solution so that only minor amount of it is left on the crystals' surface, and if there were any impurities, they don't get a chance to nucleate as their concentration in washing solvent is going to be negligible.

As for your perception of the recrystallization as of a "weird" process, I'd say it's probably more complicated than one would expect from looking at the beaker with some crystalline mess. In fact, underlying processes are described with more or less appropriate models, and once a person 
 gets experienced enough, it all starts to make much more sense.