Should the outlet that my space heater uses be getting warm?
Line voltage varies slightly by home and by region. 110v and 120v are more or less interchangeable, nominal approximations. 117v is (was?) an average. The current spec is 120v +/- 5% if I'm not mistaken.

Your heater draws a lot of current, and weak points in the circuit, such as plug connections, will often show some heat accumulation. 

It is a bit odd that other outlets would be warm as well, though. It's possible that the wiring runs through the outlet side jumpers, but that should be a sturdy connection that isn't a high-resistance point. 

Anyway, "slightly warm" isn't too concerning. Many devices will do that, such as battery chargers. When it approaches hot to the touch you have a real problem. Keep a close eye on it and try not to rely on portable heaters for long periods of time.