title: Tensorflow - Avoid Tensor Size Limit
body: <p>I'm working on an implementation of the FCN-32 net described in the <a href="https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf" rel="nofollow noreferrer">Long, Shelhamer paper</a>, but have run into a roadblock when upsampling. In order to upsample to original size, other <a href="https://github.com/MarvinTeichmann/tensorflow-fcn/blob/master/fcn32_vgg.py" rel="nofollow noreferrer">implementations</a> use a <code>conv2d_transpose</code> layer with a bilinear filter w/kernel size 64x64. This works fine until you start using lots of classes.</p>
  
  <p>For any number of classes > ~375, the <code>filters</code> variable in the transpose layer is > 2 gb ( 64 x 64 x (>375) x (>375) ) so Tensorflow complains and dies, saying</p>
  
  <p><code>ValueError: Cannot create a tensor proto whose content is larger than 2GB.
  </code></p>
  
  <p>Is there any way to avoid this size limit? My first thought would be generative tensor, but I can't find any documentation on how to create if such a construct exists or is possible.</p>