Why do I need to remove my battery periodically to fix phone problems?
It's not the battery retaining state, it's the phone circuitry retaining state. Even when you use a "power off" command and your phone looks powered off some of the circuits will still be connected to the battery so that the phone can be turned on by just pressing the button. So there will be some circuitry powered on even after the phone was "powered off".

This is likely a bug in the circuitry. This actually happens to devices - you sometimes need to "power cycle" them by unplugging them from the power source and then plugging them back. In case of a mobile phone that requires removing and then installing the battery.