How fast is the response time of a battery for a particular current demand?
The main limiting factor will be the inductance of the battery and its leads. Here's a discussion by someone who claims this is can a problem when using long leads in RC motor systems. In general inductance limits the rate of change of current flow in an electrical system.

The usual design approach is to put decoupling capacitors near the load. This also reduces noise interference in sensitive parts of the system or nearby electronics.

Note that "instantaneously" is a really short time; this isn't a problem for most systems.