TfLiteGpuDelegateBindGlBufferToTensor
`TfLiteGpuDelegateBindGlBufferToTensor` was consciously removed in v2 because we saw minor adoption due to complicated API.  Chances are very low that it'll be re-introduced.  If you want to use `TfLiteGpuDelegateBindGlBufferToTensor`, the usage above is probably not sufficient: You would have to do a lot of plumbing that `gl_delegate.cc` does (see logic around `bhwc_objects_`) and apply it to `delegate.cc`. Hi @impjdi

Thanks a lot for your kind reply.

So,, is there any other template, for v2, which takes textures as the input and output of tflite inference?

Could you please let me know if there is a v1 version (gl_delegate.cc supported) which has an opencl engine as fast as a v2 engine? I've heard that tflite engine on opencl is about twice faster than that on opengl..

I am looking forward to hearing from you. Thanks again.

Best,
Picard314  Hm, that's complicated.

> So,, is there any other template, for v2, which takes textures as the input and output of tflite inference?

Not exactly OpenGL textures, but OpenGL SSBO, you can see examples of using the internals of the GPU inference in the MediaPipe's inference_calculator_gl_advanced.cc  MediaPipe wasn't okay with the CPU / GPU copy that you're mentioning, but doesn't have the BindTensor functions, so they are using the GPU inference internal APIs directly.

> Could you please let me know if there is a v1 version (gl_delegate.cc supported) which has an opencl engine

Unfortunately, there has been no standalone OpenCL delegate IIRC.  I think when we introduced OpenCL, we merged them together into the v2 (delegate.cc). Hi @impjdi 

Ah, I am so grateful and quite shocked on your professional explanations~

If possible, I would like to request for informations (codes template) of implementing the MediaPipe from you. 

I found no "inference_calculator_gl_advanced.cc" in the source codes of tflite v2.10 I am using. Is MediaPipe a tflite branch and does it have a v2/opencl engine/delegate please?

Best,
Picard314






 > Hi @impjdi
> 
> Ah, I am so grateful and quite shocked on your professional explanations~
> 
> If possible, I would like to request for informations (codes template) of implementing the MediaPipe from you.
> 
> I found no "inference_calculator_gl_advanced.cc" in the source codes of tflite v2.10 I am using. Is MediaPipe a tflite branch and does it have a v2/opencl engine/delegate please?
> 
> Best, Picard314

MediaPipe is a different project - https://github.com/google/mediapipe/blob/master/mediapipe/calculators/tensor/inference_calculator_gl_advanced.cc
Hi, dear developers

I would like to ask if "TfLiteGpuDelegateBindGlBufferToTensor" can be used in implementing DelegateV2 (tflite version 2.10), as I hope to avoid cpu-gpu memory copy letting textures be the input and output of tflite inference.

Below is my pseudo code, could you please let me know if the implementation is correct?

#######################################################
TfLiteStatus tf_status = kTfLiteOk;
std::unique_ptr<tflite::Interpreter> tf_interpreter_;
TfLiteDelegate* tf_delegate_ = nullptr;

GLuint ssboR;
GLuint ssboW;

int net_input_tensor_index = xxx;
int net_output_tensor_index = xxx;

TfLiteGpuDelegateOptionsV2 tf_gpu_delegate_option;
tf_gpu_delegate_option = TfLiteGpuDelegateOptionsV2Default();
tf_gpu_delegate_option.inference_preference = TFLITE_GPU_INFERENCE_PREFERENCE_SUSTAINED_SPEED;
tf_gpu_delegate_option.inference_priority1 = TFLITE_GPU_INFERENCE_PRIORITY_MIN_LATENCY;

tf_delegate_ = TfLiteGpuDelegateV2Create(&tf_gpu_delegate_option);

tf_status = TfLiteGpuDelegateBindGlBufferToTensor(tf_delegate_, ssboR, net_input_tensor_index, kTfLiteFloat32, TFLITE_GPU_DATA_LAYOUT_BHWC);
tf_status = TfLiteGpuDelegateBindGlBufferToTensor(tf_delegate_, ssboW, net_output_tensor_index, kTfLiteFloat32, TFLITE_GPU_DATA_LAYOUT_BHWC);

tf_status = tf_interpreter_->ModifyGraphWithDelegate(tf_delegate_);
tf_status = tf_interpreter_->AllocateTensors();
#######################################################

Best,
Picard314



