Is recalibrating the battery after flashing a new ROM really neccessary, and if yes, why?
No.

This App just removes the file "batterystats.bin" in the "data/system" directory. It has nothing to do with your battery life and also not the status indicator of your system.

It only keeps the Infos in the "Battery Usage" screen. It will also be resetted every time you connect to a power source.

Further Information and Sources:
XDA-Developers and Dianne Hackborn (Google employee)