Questions about Li-Ion batteries
The battery has internal resistance, and what we will measure on the battery terminals should always be below 4.2(Vmax). E.g. if max voltage deliverable by battery is 4.2, we will have 0.1V internal drop, and thus measure 4.1 on battery terminals.

What you will measure across the terminals of the battery will depend upon what is happening to the battery. It may be

charging
discharging
neither supplying nor storing up power (quiescent)

When then battery is charging, the voltage across the terminals will be higher than when the battery is quiescent.
When the battery is discharging, the voltage across the terminals will be lower than when the battery is quiescent.
This is because the &quot;voltage drop of the internal resistance&quot; depends upon power flow. When there is no current and therefore no power flow, there is no &quot;voltage drop of the internal resistance&quot;. When power is flowing into the battery, the &quot;voltage drop of the internal resistance&quot; will give the terminals a higher voltage. When the power is flowing out of the battery, the &quot;voltage drop of the internal resistance&quot; will give the terminals a lower voltage.

As the battery ages the internal resistance increases, hence, after few years, I should read even less voltage on battery terminals.

That will be true when the battery is being used, i.e. discharging, and power is being drawn from the battery.

Charger IC will keep charging the battery until the terminal voltage reaches 4.2V, but it had to charge it until 4.0. So it will actually overcharge it.

No, the charger will stop at 4.2V. The quiescent voltage of the battery is only 4.0V, so the battery will not be overcharged.

Do they [i.e. low battery detectors] generate alarms according to internal or external voltage?

They generate an alarm when the terminal voltage is at a certain level. This can either be when the battery is quiescent, or in use. While quiescent, the battery voltage may be higher than when in use, but if the voltage falls below the alarm threshold while in use, it should trigger an alarm.