Batteries have zero voltage
Does this mean also that an unplugged battery has zero voltage?


No, Ohm's law is not universal; Ohm's law defines the ideal resistor circuit element.

But not all circuit elements are resistors and thus, not all circuit elements obey Ohm's law.

A battery is an approximate voltage source, not a resistor and thus does not obey Ohm's law.

For an ideal voltage source, the voltage across is independent of the current through; the voltage across is what it is regardless of the value of the current through.

Thus, if we connect an ideal voltage source and resistor together to form a circuit, the voltage across the resistor is fixed by the voltage source.

Since the voltage is fixed, the series current through the resistor and battery is determined by the resistance \$R\$ of the resistor:

$$I = \frac{1.5V}{R}$$

If we disconnect the battery, we have effectively made the resistance infinite and we get

$$I = \frac{1.5V}{\infty} = 0A$$

Physical voltage sources, such as a 1.5V battery, cannot supply unlimited current and, in fact, produce a finite current when short-circuited.

So, we typically model a physical battery by placing a resistor in series with the voltage source.  But this is the topic of another question.