title: Different predictions when running Keras model in TensorFlow Lite
body: <p>Trying out TensorFlow Lite with a pretrained Keras image classifier, I'm getting worse predictions after converting the H5 to the tflite format. Is this intended behaviour (e.g. weight quantization), a bug or am I forgetting something when using the interpreter?</p>
  
  <h1>Example</h1>
  
  <pre><code>from imagesoup import ImageSoup
  from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions
  from tensorflow.keras.preprocessing.image import load_img, img_to_array
  
  # Load an example image.
  ImageSoup().search('terrier', n_images=1)[0].to_file('image.jpg')
  i = load_img('image.jpg', target_size=(224, 224))
  x = img_to_array(i)
  x = x[None, ...]
  x = preprocess_input(x)
  
  # Classify image with Keras.
  model = ResNet50()
  y = model.predict(x)
  print("Keras:", decode_predictions(y))
  
  # Convert Keras model to TensorFlow Lite.
  model.save(f'{model.name}.h5')
  converter = tf.contrib.lite.TocoConverter.from_keras_model_file
  tflite_model = converter(f'{model.name}.h5').convert()
  with open(f'{model.name}.tflite', 'wb') as f:
      f.write(tflite_model)
  
  # Classify image with TensorFlow Lite.
  f = tf.contrib.lite.Interpreter(f'{model.name}.tflite')
  f.allocate_tensors()
  i = f.get_input_details()[0]
  o = f.get_output_details()[0]
  f.set_tensor(i['index'], x)
  f.invoke()
  y = f.get_tensor(o['index'])
  print("TensorFlow Lite:", decode_predictions(y))
  </code></pre>
  
  <blockquote>
    <p>Keras: [[('n02098105', 'soft-coated_wheaten_terrier', 0.70274395),
    ('n02091635', 'otterhound', 0.0885325), ('n02090721',
    'Irish_wolfhound', 0.06422518), ('n02093991', 'Irish_terrier',
    0.040120784), ('n02111500', 'Great_Pyrenees', 0.03408164)]]</p>
    
    <p>TensorFlow Lite: [[('n07753275', 'pineapple', 0.94529104), ('n03379051',
    'football_helmet', 0.033994876), ('n03891332', 'parking_meter',
    0.011431991), ('n04522168', 'vase', 0.0029440755), ('n02094114', 'Norfolk_terrier', 0.0022089847)]]</p>
  </blockquote>