Very old, Coleman, through the wall (double sided) mounted gas heater not working
I assume you are dealing with a gravity wall furnace, now made by Williams and Empire, and commonly found in California. They require no AC power.  They use a thermocouple on the pilot light to power (via the thermostat) a millivolt gas solenoid.  They sell special thermostats for this application.  A "common" thermostat may or may not work, depending on its ability to switch a very low voltage at comparatively high current, and function without 24v power. 

Electronic thermostats cannot work, unless you supply 24v to them and supply a relay to switch the millivolt line... but that makes the furnace dependent on AC power, which defeats the purpose of this type of furnace! 

Yes, touch red and white together, and the unit should come on.

If it does not, first check that the pilot is on (no pilot, no millivolts). Then check the 2 wires with a voltmeter, there should be some fraction of a volt of electricity.  If not, the pilot is out or you have a thermocouple problem.  

Parts are readily available for Empire and Williams furnaces, and worse comes to worst, you can replace one in-kind, though you'll have to special order it in the northeast or midwest.  For some reason, people in those areas have a revulsion to furnaces that work without electricity.