Why do batteries with the same voltage have different currents?
Current depends on Voltage". So, if the voltage is high, current would
  be high. Agreed; (I=V/R)


True, if you're asking about resistance.

But, you're asking about a (non-ideal) voltage source - a battery.

The voltage to current relationship of a battery depends on the chemistry, temperature, etc.  Cells and batteries are not resistors.

Now, it is the case that a first approximation of a battery is an ideal voltage source in series with an ideal resistor, as another answer points out, but it must be kept in mind that this effective resistance depends on a number of factors.  See, for example this.




  If two different batteries(with same voltage) delivers different
  currents, how can we say that they are both 2V batteries?


You've answered your question yourself.  They have the same open-circuit voltage.

Clearly, two identical cells connected in parallel provide the same open-circuit voltage as one of the cells but two cells in parallel can provide twice the short-circuit current of the single cell.