When you measure the potential difference across (or between) the terminals of a battery that is not in a complete circuit you are measuring the emf of the battery. This is the maximum amount of work per coulomb of charge the battery can do to drive charge from one terminal, through the circuit, to the other terminal.