Cure for burns from hot peppers / capsicum oil?
Capsaicin is oil/fat-soluble so try washing your hands with a little whole milk, or rub with sour cream or vegetable oil and see if that helps. 

Just as you can get it in your eyes if you rub them with your fingers, I'm sure you can easily transfer it to your baby.

I'm just surmising here, not speaking from experience so if you try any of these, be sure to let us know how they worked....or what you did instead.