Phenol smell from hands
As an acidic compound, phenol started to degrade the protective layers of your skin, and partially diffused into deeper layers.  This, and because of the moderate solubility in water are the reason the scent of phenol was still recognisable after some time.  While phenol is better soluble in ethanol and methanol, one refrains from these solvents to clean the skin because these actually would help phenol to reach even deeper tissues.
 Your writing that it was recognisable even after 3-4 days, even after extensive rinsing with water and soap, hints to an extended exposure of the skin towards pure phenol, which however should be avoided.

Not visible to human eye, but thanks to antiseptic power, it killed some germs, too.  Earlier in times, it was known and used as carbolic acid.