How You Burn Calories. Your weight x distance = energy used walking. Time does not matter as much as distance. If you speed up to walking a mile in 13 minutes or less, you will be burning more calories per mile. But for most beginning walkers, it is best to increase the distance before working on speed. A simple rule of thumb is 100 calories per mile for a 180 pound person.