While chalk is minimally toxic, not poisonous in small amounts, and may not hurt you, it's never a good idea to eat chalk. A pattern of eating chalk is a different story, however. Eating chalk often can disrupt your digestive system and cause damage to your internal organs.