title: How to calculate GPU memory usage in Theano?
body: <p>I am experimenting with different Theano models and use a curriculum of ever increasing sequence length.  How can I predict ahead of time how big to make the batch size for any given sequence length and model in order to fill the GPU's memory?</p>
  
  <p>To make matters worse, if I ever accidentally use too much memory, I get a MemoryError and the memory on the GPU is not freed, requiring me to restart the process to free the memory, and lose my network, before trying a new batch size.  Because this error is unrecoverable, it is difficult to just increase batch size until exception and then back down.</p>