Is high impedance important for a differential amp measuring a battery voltage?
To answer your actual question... As long as the R values are large you do not need the buffers for it to work as you imagine. However, be aware without them there will be some drain on the battery if the R values are not as large at the op-amp input impedance.

However, what concerns me more is the stack of batteries. If you plan on doing this for every battery in the stack, and the stack is large, at some point you will exceed the input voltage on the op-amp.