Why i get OOM error although my model is not that large?
1) Why I am getting OOM error on the large batch size although my dataset and model are not that big?


Yes, the batch size is probably the reason. 

Also, another the reason is that you don't use the second GPU at all (otherwise, both GPUs will split the batch - computation - and you could use larger batches).


  2) How can I utilize both GPU's equally in Tensorflow so that the performance is fast? (From the above error, it looks like GPU:0 gets full immediately whereas GPU:1 is not fully utilized. it's my understanding only)


By default, Tensorflow occupies all available GPUs (that's way you see it with nvidia-smi - you have a process 34589 which took both GPUs), but, unless you specify in the code to actually use multi GPUs, it will use only one by default.

Here is an official TF docs about how to use multi GPUs: https://www.tensorflow.org/guide/using_gpu#using_multiple_gpus

Here is some tutorial for multi-gpu usage with more examples: https://jhui.github.io/2017/03/07/TensorFlow-GPU/