Phantom Voltage When Near Outlet?
0.050 to 0.100mV when not connected to anything


This is pretty normal. It's not a big number and probably just represents the noise floor of your multimeter.


  As soon as I move close to the wall outlet, it creeps up to 0.1 volts,
  then 2 volts, then when I am pretty close to it, it reads about 4 - 6
  volts.


Also pretty normal. The electric fields emanating from house AC wiring can make a multimeter show these kind of voltages when close up. It's mainly due to the electric fields capacitively coupling to you probes with your body acting as a capacitor to ground. It would be interesting to see what the voltage drops to when you stop holding the multimeter - I would expect it to still register a number but a smaller number.


  between neutral and ground, it usually reads 0.4 volts when there is
  minimal load on the outlet. I heard that that is normal, but is 4.4
  volts from neutral to ground normal when a 1300W heater is running?


4.4 volts is certainly no un-heard of. Usually neutral is connected to earth at some point between your house and the local sub-station and given the volt drop down the neutral wire when passing several amps this would be fairly commonplace.