How to calculate the time to empty a battery
To a first approximation, yes that is how many hours. 

To get a more accurate number you have to look at the cutoff voltage (limited by the minimum voltage the device being powered can operate at, or perhaps the minimum voltage that you can discharge a rechargeable battery to before damage to the battery occurs). 

The capacity changes with the rate of discharge (usually getting worse at higher discharge rates, but at very low discharge rates the self-discharge becomes as factor). 

Temperature also plays a role- at low temperatures  capacity can be significantly less. 

Note also that it many loads do not draw even a somewhat constant current for the entire discharge time. An example of such a load would be a fixed load on a linear regulator. Since the output voltage of the regulator is fixed the load current is independent of the input voltage. Even so, many times the load will vary significantly as transmitters or motors start and stop or the device goes into sleep mode. 

A resistive load draws less current as the battery voltage drops. An incadescent filament has less variation in current as the voltage drops. A modern switching supply will draw more current as the voltage drops to maintain constant output power, and even a bit more than that since efficiency is typically lowest at minimum input voltage. 

So it's actually fairly complex  to get a real number for the battery life. One good way to confirm the numbers is to measure the battery capacity under standard datasheet conditions using a programmable digital load, and then measure the device operating time on the same battery in simulated usage.