How do I choose the optimal batch size?
Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch: 


If you have a small training set, use batch gradient descent (m &lt; 200)


In practice:


Batch mode: long iteration times
Mini-batch mode: faster learning 
Stochastic mode: lose speed up from vectorization


The typically mini-batch sizes are 64, 128, 256 or 512.

And, in the end, make sure the minibatch fits in the CPU/GPU.

Have also a look at the paper Practical Recommendations for Gradient-Based Training of Deep Architectures (2012) by Yoshua Bengio.