Why do these outlets appear to be "hot"?
If you only disconnected the neutral, it would explain why both wires show 120V. Any load anywhere in the circuit, even the miniscule vampire load of a GFCI, would "pull up" the neutral to 120V. That is normal and will happen in any circuit where neutral is interrupted. 

Is that safe?  Yes - in a system built to modern code.  Neutral and ground are isolated from each other everywhere except the main panel. (And that's why.) Neutral from circuit 1 isn't cross-connected to circuit 2 (which could overload circuit 2's neutral if it were). 

Of course old work can have a variety of defects. In modern work, there's one place in particular where neutral floating to 120V would be a problem, and that's in NEMA 10 outlets for stoves or dryers, because they treat neutral as ground and put neutral on the metal frame of the appliance.