title: Am I right to be suprised that applying a 15MB model to a 3kB image takes 1-4GB of GPU memory
body: <p>Tfhub's <a href="https://tfhub.dev/tensorflow/efficientdet/d0/1" rel="nofollow noreferrer">&quot;efficientdet d0&quot; model</a> is 15MB but when I apply it to even the smallest image, tensorflow consumes 1-4GB of GPU memory - even having <a href="https://stackoverflow.com/questions/34199233/how-to-prevent-tensorflow-from-allocating-the-totality-of-a-gpu-memory/34200194#34200194">prevented tensorflow from allocating all the GPU memory</a>. The same happens with the similarly small <a href="https://tfhub.dev/tensorflow/efficientdet/lite1/detection/1" rel="nofollow noreferrer">&quot;efficientdet lite1&quot;</a> and <a href="https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2" rel="nofollow noreferrer">&quot;ssd mobilenet v2&quot;</a> models. All three networks advertise efficient, scalable performance suitable for mobile and embedded applications.</p>
  <p>For example:</p>
  <pre><code>import tensorflow as tf
  import tensorflow_hub as hub
  
  for gpu in tf.config.list_physical_devices('GPU'):
    tf.config.experimental.set_memory_growth(gpu, True)
    # Optionally: tf.config.experimental.set_virtual_device_configuration(gpu, [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=256)])
  
  detector = hub.load(&quot;https://tfhub.dev/tensorflow/efficientdet/d0/1&quot;)
  # At this point, nvidia-smi shows 311MiB used
  
  z = tf.zeros([1,32,32,3], dtype=tf.dtypes.uint8)
  print(detector(z))
  # At this point, nvidia-smi shows 4361MiB used
  # or 1163MiB if memory_limit=256 enabled.
  </code></pre>
  <p>This came as a surprise to me - I could understand if a 15MB model was compressed and one needed 10x as much GPU memory - but needing 290x as much is a pretty big difference. 4GB here, 4GB there and pretty soon you're talking about serious GPU memory.</p>
  <p>As you can see from the code above, I'm not doing any training that would fill memory with gradients and metadata, and I'm not processing large batches or huge images.</p>
  <p>Is this normal, or am I doing something wrong here?</p>