TFLiteConverter produces model that doesn't conform to GPUv2 (TfLiteGpuDelegate Init: FULLY_CONNECTED: Amount of input channels should match weights width)
hi @gustavla ,

I tired to create the tflite model from the code you provided but i ran into this error and no tflite model was created.




---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 9
      5 output_shape = [1, 21]
      7 tf_input = keras.Input(input_shape[1:], batch_size=input_shape[0])
----> 9 tf_z = tf.reshape(tf_input, (1, 7))
     10 tf_output = tf.keras.layers.Dense(21)(tf_z)
     12 model = keras.Model(inputs=[tf_input], outputs=[tf_output])

File ~\AppData\Local\Programs\Python\Python312\Lib\site-packages\tensorflow\python\ops\weak_tensor_ops.py:88, in weak_tensor_unary_op_wrapper.<locals>.wrapper(*args, **kwargs)
     86 def wrapper(*args, **kwargs):
     87   if not ops.is_auto_dtype_conversion_enabled():
---> 88     return op(*args, **kwargs)
     89   bound_arguments = signature.bind(*args, **kwargs)
     90   bound_arguments.apply_defaults()

File ~\AppData\Local\Programs\Python\Python312\Lib\site-packages\tensorflow\python\util\traceback_utils.py:153, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    151 except Exception as e:
    152   filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153   raise e.with_traceback(filtered_tb) from None
    154 finally:
    155   del filtered_tb

File ~\AppData\Local\Programs\Python\Python312\Lib\site-packages\keras\src\backend\common\keras_tensor.py:91, in KerasTensor.__tf_tensor__(self, dtype, name)
     90 def __tf_tensor__(self, dtype=None, name=None):
---> 91     raise ValueError(
     92         "A KerasTensor cannot be used as input to a TensorFlow function. "
     93         "A KerasTensor is a symbolic placeholder for a shape and dtype, "
     94         "used when constructing Keras Functional models "
     95         "or Keras Functions. You can only use it as input to a Keras layer "
     96         "or a Keras operation (from the namespaces `keras.layers` "
     97         "and `keras.operations`). "
     98         "You are likely doing something like:\n\n"
     99         "```\n"
    100         "x = Input(...)\n"
    101         "...\n"
    102         "tf_fn(x)  # Invalid.\n"
    103         "```\n\n"
    104         "What you should do instead is wrap `tf_fn` in a layer:\n\n"
    105         "```\n"
    106         "class MyLayer(Layer):\n"
    107         "    def call(self, x):\n"
    108         "        return tf_fn(x)\n\n"
    109         "x = MyLayer()(x)\n"
    110         "```\n"
    111     )

ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces `keras.layers` and `keras.operations`). You are likely doing something like:

```
x = Input(...)
...
tf_fn(x)  # Invalid.
```

What you should do instead is wrap `tf_fn` in a layer:

```
class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)
``` This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. I wrote the repro script against tensorflow 2.15 (even though I ran on TFLite 2.16 runtime).

Model: https://qaihub-public-issues.s3.us-west-2.amazonaws.com/tflite/66729_model.tflite Hi @gustavla ,

The script that you used to create the tflite model seems to be the issue. The "input_shape" and "output_shape" should be a tuple when you are passing it to the input layer. Also because of an issue keras tensors are incompatible with tf tensors. I will suggest you to use the below script to create the model. The tflite file created with the below script should run perfectly fine using the gpu delegate.

Please run [this](https://colab.sandbox.google.com/drive/1xyYQw2WmF8MpSso4_wCcgyBKpt3B6L4y#scrollTo=G2KVTUkeaBo8) colab notebook to create your tflite model

 @sawantkumar Thanks! I tried changing to tuples, and that doesn't seem to make any difference. As for keras vs tflite tensors, I am unable to view your colab link (I requested access). Hi @gustavla ,

I have given you the access to view the colab notebook, Please review it and let me know if the tflite model created through this colab link solves your problem. TF 2.16 uses keras 3, which results in the above error. add 

```
import os
os.environ["TF_USE_LEGACY_KERAS"] = "1"
```

to the above script to make it compile
**System information**
- Compilation: Ubuntu Linux
- Execution: Google Pixel 7 / Android 14 / Google Tensor G2
- Tensorflow 2.16.1

**Standalone code to reproduce the issue** 

* Generate model with script below (note, this fails with the reshape, but the TFLiteConverter ends up with a model without the reshape)
* Run on GPUv2 on device (for instance via benchmark tool)
* Falls back to run 100% on CPU

<img width="376" alt="image" src="https://github.com/tensorflow/tensorflow/assets/902935/e0997e28-a98d-4688-bacb-2b0fc2ddd7d3">


**Generate model script**

```
import tensorflow as tf
import keras

input_shape = [7]
output_shape = [1, 21]

tf_input = keras.Input(input_shape[1:], batch_size=input_shape[0])

tf_z = tf.reshape(tf_input, (1, 7))
tf_output = tf.keras.layers.Dense(21)(tf_z)

model = keras.Model(inputs=[tf_input], outputs=[tf_output])

# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

# Save the model.
with open('model.tflite', 'wb') as f:
  f.write(tflite_model)
```

**Any other info / logs**

Runtime log (executed by https://aihub.qualcomm.com/)

<!-- job id: https://dev.aihub.qualcomm.com/jobs/j1gljz825/ -->

```
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.board.platform = gs201
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.boot.hardware = panther
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.boot.hardware.platform = gs201
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.system.build.id = UP1A.231005.007.A1
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.system.build.version.release = 14
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.hardware = panther
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.hardware.chipname = 
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.board = panther
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.brand = google
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.device = panther
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.build.fingerprint = google/panther/panther:14/UP1A.231005.007.A1/10762838:user/release-keys
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.manufacturer = Google
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.model = Pixel 7
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.product.name = panther
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.soc.manufacturer = Google
[01/May/2024:07:48:20 +10:00: profiler/info] Android system property: ro.soc.model = GS201
[01/May/2024:07:48:20 +10:00: profiler/info] [Manager] DeviceManager::DeviceManager
[01/May/2024:07:48:20 +10:00: profiler/info] [Manager] findAvailableDevices
[01/May/2024:07:48:20 +10:00: profiler/info] [Manager] Found interface google-edgetpu (version = 2.0)
[01/May/2024:07:48:20 +10:00: profiler/info] [Manager] Found interface google-armnn (version = ArmNN)
[01/May/2024:07:48:20 +10:00: profiler/info] NNAPI devices: google-edgetpu,google-armnn,nnapi-reference
[01/May/2024:07:48:20 +10:00: profiler/info] GPU device: ARM Mali-G710
[01/May/2024:07:48:20 +10:00: profiler/info] OpenGL Version: OpenGL ES 3.2 v1.r43p0-01eac0.ff5c643eda65d3f5ba9886fdffb12673
[01/May/2024:07:48:20 +10:00: profiler/info] OpenCL Version: OpenCL C 1.2 v1.r43p0-01eac0.ff5c643eda65d3f5ba9886fdffb12673
[01/May/2024:07:48:20 +10:00: profiler/info] -=- Tungsten Running Task: Loading -=-
[01/May/2024:07:48:20 +10:00: profiler/info] Detected chipset 3101, made by 3000.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Loading tflite model Models/model.tflite
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Malloc VM size before: 28856.0 kB, allocated: 18204.7 kB, slack: 10651.3 kB.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Current memory baseline range: 61424.7-72076.0 kB.
[01/May/2024:07:48:20 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Runtime metadata not found in Models/model.tflite/trt_metadata.json or Models/model.tflite/trt_metadata.pb
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] TF Lite version 2.16.1. Loading model from Models/model.tflite.
[01/May/2024:07:48:20 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Mapping resource file in Models/model.tflite
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Loaded model. Minimum TF Lite version = 1.5.0.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] No delegates specified; using compute unit=cpu_and_gpu.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Initialized TensorFlow Lite runtime.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] GPUV2 delegate requested. OpenCL detected.
[01/May/2024:07:48:20 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Enabling delegate cache in dir=/data/user/0/ai.tetra.tungsten/cache/1714513700731/ai.tetra.runtime/0.6.0/model.tflite_8945450969824422876_1714513695446/gpuv2.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Created TensorFlow Lite delegate for GPU.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Replacing 1 out of 1 node(s) with delegate (TfLiteGpuDelegateV2) node, yielding 1 partitions for the whole graph.
[01/May/2024:07:48:20 +10:00: profiler/warning] [job_id: j1p3y37mg] [model.tflite] [tflite] TfLiteGpuDelegate Init: FULLY_CONNECTED: Amount of input channels should match weights width
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Created 0 GPU delegate kernels.
[01/May/2024:07:48:20 +10:00: profiler/warning] [job_id: j1p3y37mg] [model.tflite] [tflite] TfLiteGpuDelegate Prepare: delegate is not initialized
[01/May/2024:07:48:20 +10:00: profiler/warning] [job_id: j1p3y37mg] [model.tflite] [tflite] Node number 1 (TfLiteGpuDelegateV2) failed to prepare.
[01/May/2024:07:48:20 +10:00: profiler/warning] [job_id: j1p3y37mg] [model.tflite] [tflite] Restored original execution plan after delegate application failure.
[01/May/2024:07:48:20 +10:00: profiler/error] [job_id: j1p3y37mg] [model.tflite] Delegate GPUV2/OpenCL failed to prepare. Retrying without this delegate.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] TfLite XNNPACK delegate requested.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Created TensorFlow Lite XNNPACK delegate for CPU.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Replacing 1 out of 1 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 1 partitions for the whole graph.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Applied 1 delegates: XNNPACK. Model is fully delegated=true.
[01/May/2024:07:48:20 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Saving delegate selection for subsequent steps.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Malloc VM size after: 28856.0 kB, allocated: 18272.2 kB, slack: 10583.8 kB.
[01/May/2024:07:48:20 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Status Successfully Loaded Cold with t = 16935 us and usage: before = 72076.0 kB; peakBefore = 72076.0 kB; mallocUnusedBefore = 10651.3 kB; after = 72232.0 kB; peakAfter = 72144.0 kB; mallocUnusedAfter = 10583.8 kB; increase = 0.0-223.5 kB; peak = 68.0-10719.3 kB
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Saving results to /storage/emulated/0/Android/data/ai.tetra.tungsten/files/Results/job_j1p3y37mg/job_j1p3y37mg_results.bin
[01/May/2024:07:48:21 +10:00: profiler/info] -=- Tungsten Running Task: Loading -=-
[01/May/2024:07:48:21 +10:00: profiler/info] Detected chipset 3101, made by 3000.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Loading previously saved results in /storage/emulated/0/Android/data/ai.tetra.tungsten/files/Results/job_j1p3y37mg/job_j1p3y37mg_results.bin
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Loading tflite model Models/model.tflite
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Malloc VM size before: 28336.0 kB, allocated: 17488.8 kB, slack: 10847.2 kB.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Current memory baseline range: 55556.8-66404.0 kB.
[01/May/2024:07:48:21 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Runtime metadata not found in Models/model.tflite/trt_metadata.json or Models/model.tflite/trt_metadata.pb
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] TF Lite version 2.16.1. Loading model from Models/model.tflite.
[01/May/2024:07:48:21 +10:00: profiler/debug] [job_id: j1p3y37mg] [model.tflite] Mapping resource file in Models/model.tflite
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Loaded model. Minimum TF Lite version = 1.5.0.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] TfLite XNNPACK delegate requested.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] [tflite] Replacing 1 out of 1 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 1 partitions for the whole graph.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Applied 1 delegates: XNNPACK. Model is fully delegated=true.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Malloc VM size after: 29368.0 kB, allocated: 18832.3 kB, slack: 10535.7 kB.
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Status Successfully Loaded Warm with t = 5937 us and usage: before = 66404.0 kB; peakBefore = 66404.0 kB; mallocUnusedBefore = 10847.2 kB; after = 66848.0 kB; peakAfter = 66644.0 kB; mallocUnusedAfter = 10535.7 kB; increase = 0.0-755.5 kB; peak = 240.0-11087.2 kB
[01/May/2024:07:48:21 +10:00: profiler/info] [job_id: j1p3y37mg] [model.tflite] Saving results to /storage/emulated/0/Android/data/ai.tetra.tungsten/files/Results/job_j1p3y37mg/job_j1p3y37mg_results.bin
```