title: Running Keras with double precision fails
body: <p>I am trying to run LeNet on Keras with double precision and it fails with an error: <code>TypeError: Input 'filter' of 'Conv2D' Op has type float64 that does not match type float32 of argument 'input'.</code>. The code I am using is as follows:
  </p>
  
  <pre class="lang-python prettyprint-override"><code>import numpy as np
  from sklearn.utils import shuffle
  import keras
  from keras.models import Sequential
  from keras.layers import Input, Dense, Conv2D, MaxPooling2D, Dropout,Flatten
  from keras import backend as K
  from keras.models import Model
  from keras.utils import np_utils
  import time
  import tensorflow as tf
  K.set_floatx('float64') # Note: the code works if we comment this line, i.e., with single precision
  from mlxtend.data import mnist_data
  X, y = mnist_data()
  X = X.astype(np.float64)
  X, y =  shuffle(X, y)
  keras_model = Sequential()
  keras_model.add(Conv2D(32, kernel_size=(5, 5), activation='relu', input_shape=(28,28,1), padding='same'))
  keras_model.add(MaxPooling2D(pool_size=(2, 2)))
  keras_model.add(Conv2D(64, (5, 5), activation='relu', padding='same'))
  keras_model.add(MaxPooling2D(pool_size=(2, 2)))
  keras_model.add(Flatten())
  keras_model.add(Dense(512, activation='relu'))
  keras_model.add(Dropout(0.5))
  keras_model.add(Dense(10, activation='softmax'))
  keras_model.compile(loss='categorical_crossentropy', optimizer=keras.optimizers.SGD(lr=0.01, momentum=0.95, decay=5e-4, nesterov=True))
  keras_model.fit(X.reshape((-1, 28,28, 1)), np_utils.to_categorical(y, 10), epochs=1, batch_size=64)
  </code></pre>
  
  <p>Any suggestions is much appreciated :)</p>