Why is my LSTM is working best with batch size of 2 and no hidden layers?
There are some papers which tell us that lower batch size may generalize better than large batch size. and large batch size may cause regularization in the model too. maybe that is the reason Bayesian optimization is suggesting a lower batch size for your dataset. 
Please check below papers, 

https://openreview.net/pdf?id=B1eyO1BFPr

https://openreview.net/pdf?id=B1Yy1BxCZ