Are decoupling capacitors needed with battery power?
In broad terms, you should always use them. It is simply something that cannot hurt you to do, but could cause serious problems to ignore.

You have probably not seen any major problems with your batteries because they are placed relatively close to your chips and because they have an internal resistance to snub higher frequency signals.

This could still cause power concerns in higher frequency signals. If a Microcontroller runs at 20MHz then you are having 20e6 pulses of current pulled per second. This may not seem like a big issues, but when enough inputs change at once you may cause ground bounce or many similar problems that come with high inductance paths to ground.

The wikipedia article has some background if it helps.

 Little extra on decoupling capacitor terminology

The job of a decoupling capacitor is to "decouple" your devices power draw from the rest of the circuit. If a decoupling capacitor does its job you will only measure a DC power draw. They remove the AC wave.

There are different terms for decoupling capacitors. 

The bulk capacitors act as large power sources that can supply power for periods of time, these are required for functionality. Without a bulk filter cap you will have to have time dependent current as your chip pulls power on it's cycle.

Bypass capacitors are often of lower value and are designed to terminate higher frequencies. As frequency reduces your impedance decreases for capacitors. A smaller value capacitor has a higher impedance. These small capacitors are the backbone of terminating higher frequency waves.

Decade capacitors are another term for bypass caps but the name implies more. If your bulk filter cap is .1uF then your decade caps will be .01uF and .001 and even .0001uF depending on what you are doing. Normally I only see 1 decade cap, but I have had to use 2 or 3 before.