Electromotive force of a battery
Think of a battery like this:



V1 is the voltage produced by the chemical reaction in the battery.  This is the EMF you are asking about.

However, real batteries have this annoying problem of some inevitable resistance that can be thought of as being in series with this voltage source.  If you draw some current from the battery, its voltage goes down because the current time R1 causes a voltage drop.

So how to measure V1 without R1 getting in the way?  Measure the voltage while drawing as little current as possible.  This is the open circuit voltage of the battery, and is just V1.

Even a voltmeter draws some current, but so little in a practical sense that it can be ignored.  For example, R1 might be 10s of Ohms in a small battery.  The voltmeter's 10 M&Omega; input resistance is still so large relative to that as to be infinite for our purposes.