Both prongs of an outlet carries live electricity of a switched plug
Houses of that age will not have a grounding system. Also houses that age have switched neutrals. Meaning the neutral was switched instead of the hot. Back in that time no one really saw that as a problem until it lit up enough electricians to become something that needed to be changed. 

By the way I don't think the first NEC didn't come out until somewhere around 1927. So you might want to get with someone and make plans and a budget to rework the electrical in that house because I am sure it is full of surprises.