title: How to select batch size automatically to fit GPU?
body: <p>I am training deep neural networks with a GPU. If I make samples too large, batches too large, or networks too deep, I get an out of memory error. In this case, it is sometimes possible to make smaller batches and still train.</p>
  
  <p>Is it possible to calculate GPU size required for training and determine what batch size to choose beforehand?</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>If I print network summary, it displays number of "trainable parameters". Can't I estimate from this value? For example, take this, multiply by batch size, double for gradients etc?</p>