title: The output neuron/tensor values after interpretation of a keras-to-tflite converted model are different and mis-leading compared to the original one
body: <p>I have created a CNN model using keras from tensorflow.
  The model is not accurate yet but just as a part of POC I wanted to check if the output of the same image given to the keras model and keras-to-tflite converted model are the same or not.</p>
  <p>Just to be clear, my main intension here is to make sure the output of the keras model and the converted tflite model is exactly the same.</p>
  <p>And by saying &quot;exactly the same&quot; I mean the floating point values of the output neurons/tensors of both models should be the same when the same image is fed as input image. This is what I think should happen IDEALLY.</p>
  <p>used this link to create the basic code and will modify later as needed <a href="https://www.tensorflow.org/tutorials/images/classification#create_a_dataset" rel="nofollow noreferrer">https://www.tensorflow.org/tutorials/images/classification#create_a_dataset</a></p>
  <p><strong>this is the code for creating a CNN using keras</strong></p>
  <pre class="lang-py prettyprint-override"><code>'''
  link: https://www.tensorflow.org/tutorials/images/classification#create_a_dataset
  
  '''
  
  import os
  from datetime import datetime
  
  import matplotlib.pyplot as plt
  import numpy as np
  from time import time
  import tensorflow as tf
  from tensorflow import keras
  from tensorflow.keras import layers
  from tensorflow.keras.models import Sequential
  
  
  def download_training_dataset():
      import pathlib
      dataset_url = 'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz'
      data_dir = tf.keras.utils.get_file('flower_photos', origin=dataset_url, untar=True)
      data_dir = pathlib.Path(data_dir)
  
  
  def test_image_dataset():
      sunflower_url = 'https://storage.googleapis.com/download.tensorflow.org/example_images/592px-Red_sunflower.jpg'
      sunflower_path = tf.keras.utils.get_file('Red_sunflower', origin=sunflower_url)
  
  
  def create_and_configure_datasets(img_height, img_width, batch_size):
      # It's good practice to use a validation split when developing your model.
      # Let's use 80% of the images for training, and 20% for validation.
      train_ds = tf.keras.utils.image_dataset_from_directory(
          data_dir,
          validation_split=0.2,
          subset='training',
          seed=123,
          image_size=(img_height, img_width),
          batch_size=batch_size
      )
      val_ds = tf.keras.utils.image_dataset_from_directory(
          data_dir,
          validation_split=0.2,
          subset='validation',
          seed=123,
          image_size=(img_height, img_width),
          batch_size=batch_size
      )
  
      # You can find the class names in the class_names attribute on these datasets
      class_names = train_ds.class_names
      print(class_names)
  
      # # Visualize the data
      # print('train_ds:', train_ds)
      # plt.figure(figsize=(10, 10))
      # for images, labels in train_ds.take(1):
      #     for i in range(9):
      #         ax = plt.subplot(3, 3, i + 1)
      #         print('img type:', type(images[i]), images[i].shape)
      #         print('pixel[500, 379]', images[i][500][379][0], images[i][500][379][1], images[i][500][379][2])
      #         print('pixel[379, 500]', images[i][379][500][0], images[i][379][500][1], images[i][379][500][2])
      #         print('pixel[-500, -379]', images[i][-500][-379][0], images[i][-500][-379][1], images[i][-500][-379][2])
      #         print('pixel[-379, -500]', images[i][-379][-500][0], images[i][-379][-500][1], images[i][-379][-500][2])
      #         npimg = images[i].numpy().astype(&quot;uint8&quot;)
      #         print('pixel[500, 379]', npimg[500][379][0], npimg[500][379][1], npimg[500][379][2])
      #         print('pixel[379, 500]', npimg[379][500][0], npimg[379][500][1], npimg[379][500][2])
      #         print('pixel[-500, -379]', npimg[-500][-379][0], npimg[-500][-379][1], npimg[-500][-379][2])
      #         print('pixel[-379, -500]', npimg[-379][-500][0], npimg[-379][-500][1], npimg[-379][-500][2])
      #         plt.imshow(npimg)
      #         plt.title(class_names[labels[i]])
      #         plt.axis(&quot;off&quot;)
      # exit()
  
      ################ Configure the dataset for performance ################
      AUTOTUNE = tf.data.AUTOTUNE
      train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE)
      val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)
      ################ Standardize the data ################
      normalization_layer = layers.Rescaling(1./255)
      normalized_ds = train_ds.map(lambda x, y: (normalization_layer(x), y))
      image_batch, labels_batch = next(iter(normalized_ds))
      first_image = image_batch[0]
      # Notice the pixel values are now in `[0,1]`.
      print(np.min(first_image), np.max(first_image))
  
      return train_ds, val_ds, class_names
  
  
  def create_and_compile_model(img_height, img_width, class_names, use_data_augmentation = False):
      ################ Create the model ################
      num_classes = len(class_names)
  
      ################ Data augmentation ################
      # data augmentation steps makes sure the data is fed to the model
      # 8 to 10 (by understanding) different angles(with by rotating)
      # and with different levels of zooms
      
      # the complete usability of use_data_augmentation is not done
      # currently no data_augmentation will be done(as in commenting in model creation statement)
      if use_data_augmentation:
          data_augmentation = keras.Sequential(
              [
                  layers.RandomFlip(
                      'horizontal',
                      input_shape=(img_height,img_width, 3)),
                  layers.RandomRotation(0.1),
                  layers.RandomZoom(0.1),
              ]
          )
  
      # # below is the less accurate model architecture(without dropout or data-augmentation)
      # # that leads to overfitting (acc. to example)
      # model = Sequential([
      #     layers.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
      #     layers.Conv2D(16, 3, padding='same', activation='relu'),
      #     layers.MaxPooling2D(),
      #     layers.Conv2D(32, 3, padding='same', activation='relu'),
      #     layers.MaxPooling2D(),
      #     layers.Conv2D(64, 3, padding='same', activation='relu'),
      #     layers.MaxPooling2D(),
      #     layers.Flatten(),
      #     layers.Dense(128, activation='relu'),
      #     layers.Dense(num_classes)
      # ])
  
  
      # specify data augmentation by 
      model = Sequential([
          # data_augmentation, # thinking of skipping data augmentation as the training 
          layers.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
          layers.Conv2D(16, 3, padding='same', activation='relu'),
          layers.MaxPooling2D(),
          layers.Conv2D(32, 3, padding='same', activation='relu'),
          layers.MaxPooling2D(),
          layers.Conv2D(64, 3, padding='same', activation='relu'),
          layers.MaxPooling2D(),
          layers.Dropout(0.1),
          layers.Flatten(),
          layers.Dense(320, activation='relu'),
          layers.Dense(num_classes)
      ])
  
      ################ Compile the model ################
      model.compile(
              optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy']
      )
  
      ################ Model summary ################
      model.summary()
      return model
  
  
  def train_and_save_model(model, train_ds, val_ds, model_h5_path, epochs=10):
  
      # # tensor-board requisites
      # log_dir = &quot;./tf-logs/fit/&quot; + datetime.now().strftime(&quot;%Y%m%d-%H%M%S&quot;)
      # tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
      # # tensor-board requisites
  
      # # model saving requisites
      # checkpoint_path = &quot;training_1/cp.ckpt&quot;
      # checkpoint_dir = os.path.dirname(checkpoint_path)
      # # Create a callback that saves the model's weights
      # cp_callback = tf.keras.callbacks.ModelCheckpoint(
      #     filepath=checkpoint_path,
      #     save_weights_only=True,
      #     verbose=1
      # )
      # # model saving requisites
  
      history = model.fit(
          train_ds,
          validation_data=val_ds,
          epochs=epochs,
          # callbacks=[
          #     cp_callback,
          #     # tensorboard_callback
          # ]
      )
  
      print('saving model to', model_h5_path)
      model.save(model_h5_path)
      # TODO save history here too
  
      return model
  
  
  def evaluate_model(model, validation_ds):
      model = tf.keras.models.load_model('./models/dentAndBurr_detector.h5')
      loss, acc = model.evaluate(validation_ds, verbose=2)
      print('Model - accuracy: {:5.2f}% | loss: {:5.2f}%'.format(100 * acc, 100 * loss))
  
  
  def predict_using_model(model, test_img_path, class_names):
      img = tf.keras.utils.load_img(
              test_img_path, target_size=(img_height, img_width)
      )
  
      s = time()
      img_array = tf.keras.utils.img_to_array(img)
      print(type(img_array), img_array.shape)
      print('for', test_img_path)
      print('pixel[500, 379]', img_array.item(500, 379, 0), img_array.item(500, 379, 1), img_array.item(500, 379, 2))
      img_array = tf.expand_dims(img_array, 0) # Create a batch
  
      predictions = model.predict(img_array)
      print('predictions:', predictions)
      score = tf.nn.softmax(predictions[0])
      print('score:', score)
  
      print(
          'This image most likely belongs to {} with a {:.2f} percent confidence.'
          .format(class_names[np.argmax(score)], 100 * np.max(score))
      )
      print('time taken for prediction:', time() - s, 's')
  
  
  if __name__ == '__main__':
  
      read_dataset = True
      create_model = False
      train_model = False
      eval_model = False
      reval_model = False
      read_model = True
      test_model = True
  
  
      model = None
      data_dir = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/' + \
          'datasets_scratch_detection/dataset_20211231/feat_original/ds'
  
      ################ CREATING DATA-SET ################
      # Define some parameters for the loader
      batch_size = 10
      img_height = 758
      img_width = 1072
  
      if read_dataset:
          train_ds, val_ds, class_names = create_and_configure_datasets(img_height, img_width, batch_size)
      # class_names = train_ds.class_names
  
      if create_model:
          model = create_and_compile_model(img_height, img_width, class_names)
  
      # exit() if input('continue?(y for YES anything else is a NO): ') == 'n' else print()
  
      ################ Train the model ################
  
      model_h5_path = 'models/dentAndBurr_detector.h5'
  
      if train_model:
          # calling the model training method fit
          model = train_and_save_model(model, train_ds, val_ds, model_h5_path, epochs=45)
  
      if eval_model:
          print('evaluating model to check its metrics while in memory')
          evaluate_model(model, val_ds)
  
      if read_model:
          # to check if model read after saving has changed or not
          print('revaluating model after reading from file to check its metrics are same as before saving')
          model = tf.keras.models.load_model(model_h5_path)
          if reval_model:
              evaluate_model(read_model, val_ds)
  
  
      if test_model:
          ################ Predict on new data ################
  
          # unseen notok
          # test_img_path = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/datasets_scratch_detection/' + \
          #     'dataset_20211231/feat_original/test_prediction/20211229181715.515.jpg'
  
          # seen notok
          # test_img_path = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/datasets_scratch_detection/' + \
          #     'dataset_20211231/feat_original/ds/notok/20211229183415.689.jpg'
  
          # unseen ok
          # test_img_path = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/datasets_scratch_detection/dataset_20220108/' + \
          #     '20220103_141858.321589_20220103141858.321/CAM1_20220103141858.321.jpg'
  
          test_img_path = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/' + \
              'datasets_scratch_detection/dataset_20211231/feat_original/tf-test/test_1.jpg' # file_path = filedialog.askopenfilename()
  
          predict_using_model(model, test_img_path, class_names)
  
  </code></pre>
  <p><strong>output of the above code when used to predict a specific image using the original keras model</strong></p>
  <pre class="lang-sh prettyprint-override"><code>predictions: [[0.47940558 0.8012595 ]]
  score: tf.Tensor([0.42022398 0.579776  ], shape=(2,), dtype=float32)
  This image most likely belongs to ok with a 57.98 percent confidence.
  time taken for prediction: 1.344334602355957 s
  </code></pre>
  <p><strong>this is the code for converting the above CNN to tflite model</strong></p>
  <pre class="lang-py prettyprint-override"><code>from sys import argv
  
  import tensorflow as tf
  
  
  if __name__ == '__main__':
  
      if len(argv) &lt; 3:
          print(
              'err: path to the model and saving path of converted tf-lite model is required!!\n' +
              'Usage: python tf_to_tflite_model_converter.py &lt;path-to-tf-model&gt; &lt;saving-path-of-tflite-model&gt;'
          )
          exit(0)
  
      tf_model_read_path = argv[1]
      tflite_model_save_path = argv[2]
      model = tf.keras.models.load_model(tf_model_read_path)
  
      # Convert the model.
      converter = tf.lite.TFLiteConverter.from_keras_model(model)
      tflite_model = converter.convert()
  
      # Save the model.
      with open(tflite_model_save_path, 'wb') as f:
          f.write(tflite_model)
  
  </code></pre>
  <p><strong>code for interpreting the tflite model for a specific image</strong></p>
  <pre class="lang-py prettyprint-override"><code>import tensorflow as tf
  # import tflite_runtime.interpreter as tflite
  import tkinter as tk
  from tkinter import filedialog
  import cv2 as cv
  import numpy as np
  import time
  
  
  def load_labels(filename = None):
  
      if filename is None:
          return  ['notok', 'ok']
  
      my_labels = []
      input_file = open(filename, 'r')
      for l in input_file:
          my_labels.append(l.strip())
      return my_labels
  
  
  
  # DEF. PARAMETERS
  img_row, img_column = 1072, 758
  num_channel = 3
  num_batch = 1
  input_mean = 0.
  input_std = 255.
  floating_model = False
  
  keras_model_path = &quot;./models/dentAndBurr_detector.h5.tflite&quot;
  labels_path = &quot;./models/labels_mobilenet.txt&quot;
  
  
  interpreter = tf.lite.Interpreter(keras_model_path)
  interpreter.allocate_tensors()
  
  # obtaining the input-output shapes and types
  input_details = interpreter.get_input_details()
  output_details = interpreter.get_output_details()
  print(input_details, '\n', output_details)
  
  # file selection window for input selection
  # root = tk.Tk()
  # root.withdraw()
  
  # file_path = filedialog.askopenfilename()
  file_path = 'D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/' + \
      'datasets_scratch_detection/dataset_20211231/feat_original/tf-test/test_1.jpg'
  # input_img = Image.open(file_path) # input_img = Image.open(file_path)
  input_img = cv.imread(file_path)
  input_img = cv.cvtColor(input_img, cv.COLOR_BGR2RGB)
  print(type(input_img), input_img.shape)
  print('pixel[500, 379]', input_img.item(500, 379, 0), input_img.item(500, 379, 1), input_img.item(500, 379, 2))
  
  # input_img = input_img.resize((img_row, img_column))
  input_img = np.expand_dims(input_img, axis=0)
  
  input_img = (np.float32(input_img) - input_mean) / input_std
  
  interpreter.set_tensor(input_details[0]['index'], input_img)
  
  # running inference
  interpreter.invoke()
  
  output_data = interpreter.get_tensor(output_details[0]['index'])
  print('for', file_path)
  print(output_data)
  results = np.squeeze(output_data)
  print(results)
  
  top_k = results.argsort()[-5:][::-1]
  print(results.argsort())
  print(results.argsort()[-5:])
  print(top_k)
  labels = load_labels() # load_labels(labels_path)
  for i in top_k:
      print('{0:08.6f}'.format(float(results[i] / 255.0)) + &quot;:&quot;, labels[i])
  
  
  </code></pre>
  <p><strong>output of tflite interpretation of the same image</strong></p>
  <pre class="lang-sh prettyprint-override"><code>for D:/CRM/TT_Collar_Inspection_Project/scratch_dent_detection_MLAI/datasets_scratch_detection/dataset_20211231/feat_original/tf-test/test_1.jpg
  [[0.88043416 0.18769042]]
  [0.88043416 0.18769042]
  [1 0]
  [1 0]
  [0 1]
  0.003453: notok
  0.000736: ok
  </code></pre>
  <p>for interpreting tflite I referred the code in the below stackoverflow question and also applied the change suggested in accepted answer: <a href="https://stackoverflow.com/q/56762749">.tflite model (converted from keras .h5 model) always predicts the same class with same probability</a></p>
  <p>As you can see here the the floating point values of the output layer tensors are lot different, and so much so that it is even giving me the wrong answer.</p>
  <p>One of my oberservation was that tf.keras.utils.img_to_array reads the image in RGB format so I tried converting the img from BGR to RGB using opencv but that did not work.</p>
  <p>I also tried using freeze-graph way to create freeze-graph of keras and then convert it to tflite but could not find the binary of freeze_graph or how to run it using the python file in tensorflow/python/tools folder in venv.<br>
  This is the answer that suggests the use of freeze-graph <a href="https://stackoverflow.com/a/51881268">https://stackoverflow.com/a/51881268</a></p>
  <p>Please let me know what should be done here, is there something that I am missing or have not understood things properly.</p>
  <p>Tensorflow version: 2.7.0</p>
  <p>FYI, I have only scratched the surface of Machine Learning &amp; Deep Learning so do not have all the required knowledge.</p>
  <p>Thanks in Advance!!</p>