Can you turn on a light bulb using only one of the two wires of the wall outlet?
If it's a very tiny lightbulb that doesn't require much power at all, then yes it can work. It works because the 2nd wire from the lamp is connected by parasitic capacitance to electrical ground, completing the circuit. There's a type of testers used to check if an outlet is "hot" that work on this principle. They work best when the user touches the 2nd wire to increase the capacitance to ground.


  Will this also happen with a high 120V DC battery?


No, because a capacitive connection can only pass AC current.


  If this where to also be true with the battery this will only work by connecting the positive side of the battery correct? If you where to touch the negative side of the battery nothing will happen correct?


If you want to power a circuit from a battery you have to connect one side of the circuit to the positive terminal of the battery and the other side of the terminal to the negative terminal of the battery. 

You can't power a circuit by connecting either one of the terminals on its own to the circuit.