title: Why does this Keras model require over 6GB of memory?
body: <p>This Keras model seems to require 6GB+ of RAM using the Tensorflow backend. My back-of-the-envelope math suggests that storing the weights shouldn't require more than 500MB. What's going on?</p>
  
  <pre><code>from keras.models import Sequential
  from keras.layers.core import Dense, Activation, Dropout, Flatten
  from keras.layers.convolutional import Convolution2D, MaxPooling2D
  
  IMAGE_SIZE = 128
  print('Build model...')
  model = Sequential()
  # three color channels, 128x128
  # 16 con filters, 3 rows, 3 columns
  model.add(Convolution2D(16, 3, 3, input_shape=(3, IMAGE_SIZE, IMAGE_SIZE)))
  model.add(Activation('relu'))
  model.add(Flatten())
  model.add(Dense(1))
  model.add(Dense(3 * IMAGE_SIZE * IMAGE_SIZE))
  
  
  model.compile(loss='mse', optimizer='sgd')
  </code></pre>
  
  <p>It's a convolution layer (16 3x3 filters) connected to a single neuron, and then that single neuron is connected to ~50k neurons. </p>
  
  <p>I'm pretty new to Keras, so I imagine my misunderstanding is pretty fundamental, but I can't seem to figure it out.</p>