Why the voltage in a circuit changes when the battery has a resistance but not when the battery doesn't has a resistance?
All real batteries have internal resistance. When they deliver a current, their terminal voltage drops. A battery with no internal resistance is a model, an abstraction, that can be convenient to think about.

Some batteries have such a low internal resistance, that if you draw a modest current, you might not notice any change at all with a normal meter. For instance, drive a few LEDs with a car battery or a LiPo. Drive the same LEDs from a coin cell, and you will see a big change in output voltage.

Car batteries and LiPos do have a finite internal resistance however, if you short circuit them, the current that flows is not infinite, just too big for safety, possibly in the 100s of amps.