how to download or install .so file for tflite conversion with gpu delegate
concrete_func = model_beam_search.__call__.get_concrete_function()# Create a TFLite converter and set the delegate to TfLiteGpuDelegateconverter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func], model_beam_search)converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]converter.optimizations = [tf.lite.Optimize.DEFAULT]converter.target_spec.supported_types = [tf.float16]# Replace TfLiteFlexDelegate with TfLiteGpuDelegategpu_delegate = tf.lite.experimental.load_delegate('libtensorflowlite_gpu_delegate.so')converter.experimental_new_converter = True  # This flag is needed for using the experimental converterconverter.experimental_new_quantizer = False  # You can enable quantization if neededconverter.experimental_enable_resource_variable = False  # You can enable resource variables if neededconverter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, gpu_delegate]tflite_model = converter.convert()# Save the TFLite model to a filewith open('testing_gpu.tflite', 'wb') as f:    f.write(tflite_model) @Alwaysadil Could you please let us know which TF version you are using here and refer to this GPU [delegate](https://www.tensorflow.org/lite/performance/gpu) guide for more information on this. Thank you! @sushreebarsa I am using 2.13.0 @Alwaysadil Thank you for your quick response!
Could you please let us know if the GPU delegate guide helped you anyway. Thank you! @sushreebarsa yes it was helpful for me that u shared GPU delegate documentation, thank you for sharing that ,but i didn't understood how to get .so file libtensorflowlite_gpu_delegate.so Hi @Alwaysadil 

To get `libtensorflowlite_gpu_delegate.so`, you have to build the `.so` file using following steps

1. Install bazel 5.3.0
2. git clone https://github.com/tensorflow/tensorflow.git
3. cd tensorflow
4. git checkout r2.13
5. ./configure (yes for android build and provide the NDK and SDK versions)
6. Run `bazel build --config android_arm64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so`

Please refer to this [documentation](https://www.tensorflow.org/lite/android/delegates/gpu_native#enable_gpu_acceleration) for reference.

Thanks.
 @pjpratik could you please provide me the google colab notebook with code to get .so file please? i'm getting errors in google colab (please check my google colab )

https://colab.research.google.com/drive/1aaX-Dm_TySAWWWyR1S6UEiPc5EB9kPjQ#scrollTo=nhrzFEC7GDXr

like this WARNING: Target pattern parsing failed.
Loading: 0 packages loaded
    currently loading: tensorflow/lite/delegates/gpu
ERROR: no such package '@local_config_tensorrt//': Repository command failed
Could not find any NvInferVersion.h matching version '' in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
        'local/cuda/extras/CUPTI/include'
of:
        '/lib'
        '/lib/x86_64-linux-gnu'
        '/lib32'
        '/usr'
        '/usr/local/cuda'
        '/usr/local/cuda/targets/x86_64-linux/lib'
        '/usr/local/lib'
Analyzing: 0 targets (0 packages loaded)
    currently loading: tensorflow/lite/delegates/gpu
INFO: Elapsed time: 83.761s
Analyzing: 0 targets (0 packages loaded)
    currently loading: tensorflow/lite/delegates/gpu
INFO: 0 processes.
Analyzing: 0 targets (0 packages loaded)
    currently loading: tensorflow/lite/delegates/gpu
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tensorflow/lite/delegates/gpu
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tensorflow/lite/delegates/gpu
    Fetching @local_config_rocm; fetching


plese help me Hi @Alwaysadil 

The colab shared is currently not accessible. Could you please provide the steps you have followed?

You can follow these [instructions](https://www.tensorflow.org/lite/android/lite_build#install_bazel_and_android_prerequisites) along with links to download  for  setting up the configurations in your local machine.  Please let us know if you are facing any issue after the following the steps.

Thanks. Hi @pjpratik thank you for your response.Could you please check my colab notebook it will now accessible,i'm unable to downloading the .so file please kindly go through this colab link https://colab.research.google.com/drive/1aaX-Dm_TySAWWWyR1S6UEiPc5EB9kPjQ#scrollTo=nhrzFEC7GDXr

i want to load this with tf 
import tensorflow as tf
delegate = tf.lite.experimental.load_delegate('libtensorflowlite_gpu_delegate.so')#with this we can get faster predictions of tflite model

please help me 

  Hi @Alwaysadil 

Thanks for sharing the code. I can see that the android ndk and sdk tools have not been configured. 

The Android NDK is required to build the native (C/C++) TensorFlow Lite code. The current recommended version is 21e, which may be found [here](https://developer.android.com/ndk/downloads/older_releases.html#ndk-21e-downloads).
The Android SDK and build tools may be obtained [here](https://developer.android.com/tools/revisions/build-tools.html).

Run the ./configure script in the root TensorFlow checkout directory, and answer "Yes" when the script asks to interactively configure the ./WORKSPACE for Android builds. 

Also, you can this prebuilt `.so` file and see if it works for your case.
[libtensorflowlite_gpu_delegate.so.zip](https://github.com/tensorflow/tensorflow/files/12488492/libtensorflowlite_gpu_delegate.so.zip)

Thanks. Hi @pjpratik 
Thanks for you response it means alot ,

ould you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y
Searching for NDK and SDK installations.

Please specify the home path of the Android NDK to use. [Default is /root/Android/Sdk/ndk-bundle]: 


The path /root/Android/Sdk/ndk-bundle or its child file "source.properties" does not exist.
Please specify the home path of the Android NDK to use. [Default is /root/Android/Sdk/ndk-bundle]: /content/android-ndk-r21e


Please specify the (min) Android NDK API level to use. [Available levels: ['16', '17', '18', '19', '21', '22', '23', '24', '26', '27', '28', '29', '30']] [Default is 26]: 


Please specify the home path of the Android SDK to use. [Default is /root/Android/Sdk]: /root/android-sdk


Please specify the Android SDK API level to use. [Available levels: ['30']] [Default is 30]: 


Please specify an Android build tools version to use. [Available versions: ['30.0.3']] [Default is 30.0.3]: 


Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=mkl_aarch64 	# Build with oneDNN and Compute Library for the Arm Architecture (ACL).
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
	--config=v1          	# Build with TensorFlow 1 API instead of TF 2 API.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=nogcp       	# Disable GCP support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished
https://colab.research.google.com/drive/1aaX-Dm_TySAWWWyR1S6UEiPc5EB9kPjQ#scrollTo=nhrzFEC7GDXr
**but the .so file was not visible ,i downloaded it by giving path like this see above colab link the way i did**
#checking the path
import os

path_to_check = '/content/tensorflow/bazel-bin/tensorflow/lite/delegates/gpu/libtensorflowlite_gpu_delegate.so'

if os.path.exists(path_to_check):
    print("yes")
else:
    print("no")
**it was printing yes**

#downloading the .so file 
from google.colab import files

source_path = '/content/tensorflow/bazel-bin/tensorflow/lite/delegates/gpu/libtensorflowlite_gpu_delegate.so'

if os.path.exists(source_path):
    files.download(source_path)
else:
    print("Source file does not exist.")



 i have successfully downloaded the  libtensorflowlite_gpu_delegate.so see this below link to get newly built .so file  https://drive.google.com/file/d/1848HQ4ExO72zkTdQC-yr7rrc7kvVfeeE/view?usp=sharing

**while loading this .so in new colab notebook  file i am getting this below error for both newly built and you shared prebuild .so file**check this https://colab.research.google.com/drive/1jAaFDTwqRWuISD0nA6OF9d0eSq39t6w1?usp=sharing

import tensorflow as tf
delegate = tf.lite.experimental.load_delegate(''/content/drive/MyDrive/delegate/libtensorflowlite_gpu_delegate.so')#with this we ca get faster predictions
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-6-b8c40a9955bd> in <cell line: 2>()
      1 import tensorflow as tf
----> 2 delegate = tf.lite.experimental.load_delegate(''/content/drive/MyDrive/delegate/libtensorflowlite_gpu_delegate.so')#with this we ca get faster predictions

3 frames
/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/interpreter.py in load_delegate(library, options)
    164   """
    165   try:
--> 166     delegate = Delegate(library, options)
    167   except ValueError as e:
    168     raise ValueError('Failed to load delegate from {}\n{}'.format(

/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/interpreter.py in __init__(self, library, options)
     71                          'due to missing immediate reference counting.')
     72 
---> 73     self._library = ctypes.pydll.LoadLibrary(library)
     74     self._library.tflite_plugin_create_delegate.argtypes = [
     75         ctypes.POINTER(ctypes.c_char_p),

/usr/lib/python3.10/ctypes/__init__.py in LoadLibrary(self, name)
    450 
    451     def LoadLibrary(self, name):
--> 452         return self._dlltype(name)
    453 
    454     __class_getitem__ = classmethod(_types.GenericAlias)

/usr/lib/python3.10/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    372 
    373         if handle is None:
--> 374             self._handle = _dlopen(self._name, mode)
    375         else:
    376             self._handle = handle

OSError: libtensorflowlite_gpu_delegate.so: cannot open shared object file: No such file or directory


**but if i check the path**
import os

path_to_check = '/content/drive/MyDrive/delegate/libtensorflowlite_gpu_delegate.so'

if os.path.exists(path_to_check):
    print("yes")
else:
    print("no")

#it was printing "yes"


**please help me to overcome this issue**
 Hi @Alwaysadil 

Apologies for the confusion. The delegate can be loaded only if it matches the target architecture. The colab ships with `x86_64`(test it out using `!uname -a`), hence we need to build the  `bazel build --config android_x86_64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so` and then try to load the delegate.

Thanks. Hi @pjpratik i didn't get what u said,i tried in local system terminal too still same error occurring, could you please help me to overcome this issue Hi @Alwaysadil 

The `libtensorflow_gpu_delegate.so` is built with respect to the target configuration specified during the bazel build. Please check you system target architecture. On linux machine simply, run 

```
bazel build -c opt tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so --copt -DEGL_NO_X11=1

```
instead of 
```
bazel build --config android_arm64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so
```
Additionally, you may have to install
```
sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev
sudo apt-get install mesa-utils
```

Thanks. > Hi @Alwaysadil
> 
> Apologies for the confusion. The delegate can be loaded only if it matches the target architecture. The colab ships with `x86_64`(test it out using `!uname -a`), hence we need to build the `bazel build --config android_x86_64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so` and then try to load the delegate.
> 
> Thanks.

hi @pjpratik, still getting this below error ,the .so file was not visible tf.lite.experimental.load_delegate("/content/drive/MyDrive/test_delegate/libtensorflowlite_gpu_delegate.so")
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-8-e59bb353aedb> in <cell line: 1>()
----> 1 tf.lite.experimental.load_delegate("/content/drive/MyDrive/test_delegate/libtensorflowlite_gpu_delegate.so")

3 frames
/usr/lib/python3.10/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    372 
    373         if handle is None:
--> 374             self._handle = _dlopen(self._name, mode)
    375         else:
    376             self._handle = handle

OSError: libEGL.so: cannot open shared object file: No such file or directory
https://colab.research.google.com/drive/1aaX-Dm_TySAWWWyR1S6UEiPc5EB9kPjQ#scrollTo=nhrzFEC7GDXr

!uname -a
it prints---> #Linux a8fc57c0d917 5.15.109+ #1 SMP Fri Jun 9 10:57:30 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
!bazel build --config android_x86_64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so 

path_to_check = '/content/drive/MyDrive/test_delegate/libtensorflowlite_gpu_delegate.so'

if os.path.exists(path_to_check):
print("yes")
else:
print("no")

#it was printing "yes"

**please help me to get the .so file file please**  > Hi @Alwaysadil
> 
> The `libtensorflow_gpu_delegate.so` is built with respect to the target configuration specified during the bazel build. Please check you system target architecture. On linux machine simply, run
> 
> ```
> bazel build -c opt tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so --copt -DEGL_NO_X11=1
> ```
> 
> instead of
> 
> ```
> bazel build --config android_arm64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so
> ```
> 
> Additionally, you may have to install
> 
> ```
> sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev
> sudo apt-get install mesa-utils
> ```
> 
> Thanks.

Hi @pjpratik in terminal i am getting this error below dated.
ERROR: An error occurred during the fetch of repository 'llvm-raw':
   Traceback (most recent call last):
	File "/home/sys2022/tensorflow/third_party/repo.bzl", line 73, column 33, in _tf_http_archive_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz to /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012: write (No space left on device)
ERROR: /home/sys2022/tensorflow/WORKSPACE:11:14: fetching _tf_http_archive rule //external:llvm-raw: Traceback (most recent call last):
	File "/home/sys2022/tensorflow/third_party/repo.bzl", line 73, column 33, in _tf_http_archive_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz to /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012: write (No space left on device)
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/7d879c8b161085a4374ea481b93a52adb19c0529.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: no such package '@llvm-raw//utils/bazel': java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24dINFO: Repository 'llvm-raw' used the following cache hits instead of downloading the corresponding file.
 * Hash '3e91127af59a6b07fea7901c80a7b8b9234eced42b0f14abbad5f9f7674dba69' for https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz
If the definition of 'llvm-raw' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'llvm-raw':
   Traceback (most recent call last):
	File "/home/sys2022/tensorflow/third_party/repo.bzl", line 73, column 33, in _tf_http_archive_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz to /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012: write (No space left on device)
ERROR: /home/sys2022/tensorflow/WORKSPACE:11:14: fetching _tf_http_archive rule //external:llvm-raw: Traceback (most recent call last):
	File "/home/sys2022/tensorflow/third_party/repo.bzl", line 73, column 33, in _tf_http_archive_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz to /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012: write (No space left on device)
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/7d879c8b161085a4374ea481b93a52adb19c0529.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: no such package '@llvm-raw//utils/bazel': java.io.IOException: Error extracting /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz to /home/sys2022/.cache/bazel/_bazel_sys2022/49a37d0bef434a2dcf9fbacaf7083ab7/external/llvm-raw/temp13989436987824143012: write (No space left on device)
INFO: Elapsed time: 13.339s
INFO: 0 processes.
 ,**but please provide me code or any other way  in google colab i'm comfortable with colab please help me to overcome this** **issue** hi @pjpratik 

delegate = tf.lite.experimental.load_delegate('/content/tensorflow/bazel-bin/tensorflow/lite/delegates/gpu/libtensorflowlite_gpu_delegate.so')#with this we ca get faster predictions
OSError: libEGL.so: cannot open shared object file: No such file or directory

please help me to get .so file Ensure that the directory containing libEGL.so is included in your LD_LIBRARY_PATH environment variable. You can check the current value of LD_LIBRARY_PATH by running:

echo $LD_LIBRARY_PATH
If the directory is not included, you can add it by modifying your shell profile configuration file (e.g., ~/.bashrc or ~/.bash_profile) and adding the following line:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/directory
Replace /path/to/directory with the actual path to the directory containing libEGL.so. Then, reload your shell or run source ~/.bashrc to apply the changes. > Ensure that the directory containing libEGL.so is included in your LD_LIBRARY_PATH environment variable. You can check the current value of LD_LIBRARY_PATH by running:
> 
> echo $LD_LIBRARY_PATH If the directory is not included, you can add it by modifying your shell profile configuration file (e.g., ~/.bashrc or ~/.bash_profile) and adding the following line:
> 
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/directory Replace /path/to/directory with the actual path to the directory containing libEGL.so. Then, reload your shell or run source ~/.bashrc to apply the changes.

thanks for your response,could you please tell me how to overcome this issue in google colab ?
 Hello @pjpratik , could you please reply to me? Hi @Alwaysadil 

We are actively working it. 

The TFlite GPU delegates are primarily build for the GPU backend support for Android and iOS. For ubuntu machines it is not well supported and tested.

Depending on the platform and version, we may have to define MESA_EGL_NO_X11_HEADERS and/or EGL_NO_X11 which don't include the X11 headers named slightly differently. We need to dig into EGL header files a little bit.

Thanks. Hi @pjpratik thank you for responding  
could you please tell me, 
At android side, for keras_nlp transformer models GPU  delegate will apply or not ?
i am working on this gpu delegation applying at android side  from past two months to get fatser inference from tflite model,but i am getting this error  **(java.lang.IllegalArgumentException: Internal error: Error applying delegate:)** **but,for image related tflite models the GPU Delegate was appying perfectly without any errors, but not for Keras_nlp transformers,**
why for keras_nlp transformer tflite models GPU was not applying? This was my main issue **please give me one reply about this please.....**


**I read that when performing TFLite conversion in Python programming, including a GPU delegate like the one below can lead to faster output predictions, resulting in lower latency(is it right or wrong?)**

concrete_func = model_beam_search.__call__.get_concrete_function()
converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func], model_beam_search)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.SELECT_TF_OPS]
converter.experimental_new_converter=tf.lite.experimental.load_delegate('libtensorflowlite_gpu_delegate.so')

dynamic_quant = converter.convert()

https://colab.research.google.com/drive/1GJ8UBa8GQBSKY4NC6_tYB0uxujvUn2kT?usp=sharing#scrollTo=C4iu7JIanl5d

that's why i ask you to how to download -->libtensorflowlite_gpu_delegate.so

**could you please help me to overcome this issue?** Hi @pjpratik please,give me one reply Hi @Alwaysadil 

Sorry for the delayed response. 

There are some limitations to what TensorFlow ML operations, or ops, can be accelerated by the TensorFlow Lite GPU delegate. Not all Ops currently support using GPU delegate. If the model has Ops that does not support the use of GPU delegate, you might encounter this error.

Please check this list of Ops supported [here](https://www.tensorflow.org/lite/performance/gpu#supported_ops).

>I read that when performing TFLite conversion in Python programming, including a GPU delegate like the one below can lead to faster output predictions, resulting in lower latency(is it right or wrong?)

Yes, enabling use of GPUs with your TensorFlow Lite ML applications can provide the benefits like speed and efficiency. But as mentioned earlier, it is platform dependent and we need to build the delegates accordingly, to make the best use of them.

Also,  the GPU delegate has some restrictions with the batch dimensions and requires that dimension to be consistent throughout the network.

BTW,  you can use Model Analyzer API to check which node has the compatibility issue.
```
import tensorflow as tf

tf.lite.experimental.Analyzer.analyze(model_path='model.tflite', gpu_compatibility=True)

```

Thanks. Hi @pjpratik 
Thanks for your response
i already check this **tf.lite.experimental.Analyzer.analyze(model_path='model.tflite', gpu_compatibility=True)**
i am getting GPU COMPATABILITY WARNINGS 
Subgraph#0 main(T#0) -> [T#31]
  Op#0 EXPAND_DIMS(T#0, T#13[1]) -> [T#15]
GPU COMPATIBILITY WARNING: Not supported op EXPAND_DIMS
  Op#1 TILE(T#15, T#7[1, 4, 1]) -> [T#16]
  Op#2 SHAPE(T#0) -> [T#17]
GPU COMPATIBILITY WARNING: Not supported op SHAPE
  Op#3 STRIDED_SLICE(T#17, T#9[1], T#10[0], T#9[1]) -> [T#18]
GPU COMPATIBILITY WARNING: STRIDED_SLICE supports for 3 or 4 dimensional tensors only.
  Op#4 CONCATENATION(T#5[4], T#18) -> [T#19]
  Op#5 RESHAPE(T#16, T#19) -> [T#20]
GPU COMPATIBILITY WARNING: Expected 1 runtime input tensor(s), but node has 2 runtime input(s).
  Op#6 WHILE(T#11[0], T#4[2, 0, 0, 0, 0, ...], T#13[1], T#14, T#20, Cond: Subgraph#1, Body: Subgraph#2) -> [T#21, T#22, T#23, T#24, T#25]
GPU COMPATIBILITY WARNING: Not supported op WHILE
  Op#7 RESHAPE(T#22, T#8[1, 4, 15]) -> [T#26]
  Op#8 RESHAPE(T#24, T#1[1, 4]) -> [T#27]
  Op#9 TOPK_V2(T#27, T#6[4]) -> [T#28, T#29]
GPU COMPATIBILITY WARNING: Not supported op TOPK_V2
  Op#10 CAST(T#29) -> [T#30]
GPU COMPATIBILITY WARNING: Not supported Cast case. Input type: INT32 and output type: INT64
  Op#11 GATHER(T#26, T#30) -> [T#31]
GPU COMPATIBILITY WARNING: Does not accept INT32 input.


GPU COMPATIBILITY WARNING: Subgraph#0 has GPU delegate compatibility issues at nodes 0, 2, 3, 5, 6, 9, 10, 11 with TFLite runtime version 2.13.0

please check this colab

https://colab.research.google.com/drive/1GJ8UBa8GQBSKY4NC6_tYB0uxujvUn2kT#scrollTo=vKdwHlNHLI6Y

**please tell me how to overcome this issue ,  to apply  the GPU Delegate at android side please to get fatser inference** Hi @Alwaysadil 

Given the GPU compatibility warnings, I'm afraid we cannot leverage the GPU delegate to its extent. 

I see a similar issue #59232.

On Android devices, you can also try [NNAPI delegate](https://www.tensorflow.org/lite/android/delegates/nnapi#trying_the_nnapi_delegate_on_your_own_model) which also helps in accelerating the code and let us know if it works for your case?

Thanks. Thank you @pjpratik for responding
While applying  NNAPI delegate ,getting this below error 


DeviceManager::DeviceManager2023-09-06 14:17:38.049 findAvailableDevices2023-09-06 14:17:38.104 Found interface xtensa-xtensa (version = Xtensa ANN 1.2)2023-09-06 14:17:38.104 Created TensorFlow Lite delegate for NNAPI.2023-09-06 **14:17:39.685 java.lang.IllegalArgumentException: Internal error: Error applying delegate: 2023-09-06 14:17:39.687**     at org.tensorflow.lite.NativeInterpreterWrapper.createInterpreter(Native Method)2023-09-06 14:17:39.688     at org.tensorflow.lite.NativeInterpreterWrapper.init(NativeInterpreterWrapper.java:110)2023-09-06 14:17:39.689     at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:73)2023-09-06 14:17:39.690     at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)2023-09-06 14:17:39.691     at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)

**please @pjpratik help me to overcome this speed related issues please........**
 Hi @Alwaysadil, I think we need more context.. if you can export the android studio project or a toy version which exhibits the behavior, that'll be the easiest for us to investigate. If not please at least at least post the code around where you are applying and using the NNAPI delegate, so that we may attempt to reproduce your issue. Thanks! Hi @pkgoogle thank you for responding
see this below code

#NNAPI delegate
ByteBuffer buffer = loadModelFile(mContext.getAssets());            Interpreter.Options options = (new Interpreter.Options());            NnApiDelegate nnApiDelegate = null;// Initialize interpreter with NNAPI delegate for Android Pie or above            if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) {                nnApiDelegate = new NnApiDelegate();                options.addDelegate(nnApiDelegate);            }// Initialize TFLite interpreter            try {                tflite = new Interpreter(buffer, options);            } catch (Exception e) {                throw new RuntimeException(e);            }



#GPU delegate applying

ByteBuffer buffer = loadModelFile(mContext.getAssets());            Interpreter.Options opt = new Interpreter.Options();// Initialize interpreter with GPU delegate            CompatibilityList compatList = new CompatibilityList();            if(compatList.isDelegateSupportedOnThisDevice()){                // if the device has a supported GPU, add the GPU delegate                GpuDelegateFactory.Options options1 = new GpuDelegateFactory.Options().setQuantizedModelsAllowed(true);                GpuDelegate gpuDelegate = new GpuDelegate(options1);                opt.addDelegate(gpuDelegate);            } else {                // if the GPU is not supported, run on 4 threads                opt.setNumThreads(4);            }//            opt.setNumThreads(4);            tflite = new Interpreter(buffer, opt); Hi @Alwaysadil, which model file does your "mContext.getAssets()" refer to? There's a couple of possible ones in the various colabs you shared, so I want to just focus on the one that is causing the issue.
### 1. System information

- OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
- TensorFlow installation (pip package or built from source):
- TensorFlow library (version, if pip package or github SHA, if built from source):

### 2. Code

Provide code to help us reproduce your issues using one of the following options:

#### Option A: Reference colab notebooks

1)  Reference [TensorFlow Model Colab](https://colab.research.google.com/gist/ymodak/e96a4270b953201d5362c61c1e8b78aa/tensorflow-datasets.ipynb?authuser=1): Demonstrate how to build your TF model.
2)  Reference [TensorFlow Lite Model Colab](https://colab.research.google.com/gist/ymodak/0dfeb28255e189c5c48d9093f296e9a8/tensorflow-lite-debugger-colab.ipynb): Demonstrate how to convert your TF model to a TF Lite model (with quantization, if used) and run TFLite Inference (if possible).

```
(You can paste links or attach files by dragging & dropping them below)
- Provide links to your updated versions of the above two colab notebooks.
- Provide links to your TensorFlow model and (optionally) TensorFlow Lite Model.
```

#### Option B: Paste your code here or provide a link to a custom end-to-end colab

```
(You can paste links or attach files by dragging & dropping them below)
- Include code to invoke the TFLite Converter Python API and the errors.
- Provide links to your TensorFlow model and (optionally) TensorFlow Lite Model.
```

### 3. Failure after conversion
If the conversion is successful, but the generated model is wrong, then state what is wrong:

- Model produces wrong results and/or has lesser accuracy.
- Model produces correct results, but it is slower than expected.

### 4. (optional) RNN conversion support
If converting TF RNN to TFLite fused RNN ops, please prefix [RNN] in the title.

### 5. (optional) Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
