Is the hairy skin below a chestnut’s tough skin poisonous?
There are two reasons chestnuts are usually eaten fully peeled:


The thin fuzzy membrane can be bitter, especially in raw nuts. Cooking and roasting mellows it a bit.
The membrane is hard, fibrous and unpleasant, especially compared with the soft interior.


Note that the skin is not poisonous, but depending on the intended use1 of the chestnuts, it’s probably a good idea to really take the time to clean the nuts properly. If prepared right and still hot, the skin should come off easily. 



1 Think purée, desserts or marrons glacés. If I buy roasted chestnuts on the street, I just peel as good as I can and don’t bother with tiny pieces of skin.