Accuracy issue during elementwise layer broadcasting in TFlite gpu delegate
Regarding the model accuracy for model_3 : (1, 10, 1) x (1, 10, 3) , I think updating this condition for 3d shapes will resolve it.

https://github.com/tensorflow/tensorflow/blob/1b8f5c396f0c016ebe81fe1af029e6f205c926a4/tensorflow/lite/delegates/gpu/common/model_builder.cc#L814

It would be helpful if there can be clarity on broadcasting of (1, 10, 1) and (3). Hi @gokulkrishna98 Thanks for reporting the issue.

Could you please share a reproducible code/gist in order to expedite trouble shooting process.

Thanks. Hi @pjpratik,
I am doing inferencing of mul layer  of shape (1, 1600, 1) x (384) via TFliteGpuDelegateV2 through samsung galaxy s22.
using the steps mentioned here: https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_c
and https://www.tensorflow.org/lite/performance/gpu#delegate_serialization

```
// Load the model
std::unique_ptr<tflite::FlatBufferModel> model =
    tflite::FlatBufferModel::BuildFromFile(filename);

// Build the interpreter
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);

// Resize input tensors, if desired.
interpreter->AllocateTensors();

float* input = interpreter->typed_input_tensor<float>(0);
// Fill `input`.

interpreter->Invoke();

float* output = interpreter->typed_output_tensor<float>(0);
```

There is accuracy issue during this inferencing when setting tfliteGpuDelegateV2 instead of default xnnpack (cpu). The accuracy issue report can be seen in my previous message. @gokulkrishna98 Thanks for the information.

Have you tried on TF 2.11 and TF Nightly and see if the issue still exists?

Thanks. Yes, I have also tested it on tf2.11, the same issue persists. I believe there has been no change in elementwise parser by commit history in nightly, so I don’t think testing is required there. Also, https://github.com/tensorflow/tensorflow/issues/60043#issuecomment-1476327924,

 here the input tensor 1 should be the greater one, but the channel size is checked for equality  @gokulkrishna98 Thanks for the information.

@sachinprasadhs Could you please look into this issue? Thanks. I'm not entirely clear what you mean by `{input1 - [1, 1600, 1] and input2 - [384]}` (this is not even mathematically well-formed?), but note that we're quite ad hoc when it comes to broadcast, because whatever seems so natural and the norm in Python doesn't necessarily mean that's the norm in C++.  In fact, all the broadcast situations have to be hand-coded in an ad-hoc fashion.  Having said that, we assume (probably not the best assumption) that the left operand is "bigger" and the right operand is "smaller", and that's not even universally true; I think we support that in ADD but not necessarily in MUL.  You have to inspect the code; it's been a while since I last checked it.  IIRC, the logic used to be something like:

```
if (right operand shape > left operand shape) {
  swap(left, right);  // so that right is always bigger, and the right is always smaller
}

where > is defined as, 

return l.b > r.b || l.h > r.h || l.w > r.w || l.c > r.c;  // C operator precedence to be reflected
```

now, your tensor (1, 1600, 1) will be probably expanded to (1, 1, 1600, 1) and the 384 will be expanded to (1, 1, 1, 384).  Due to the definition of >, (1, 1600, 1) will be probably considered as a bigger tensor (because 1600 gets compared to 1 before 1 > 384 is evaluated) and the 384 the smaller tensor.  Now... then comes one of the bad assumptions.  We use the dimension of the bigger tensor to do the math, so ... we will probably only use (1, 1, 1600, 1) and the 1st value of (1, 1, 1, 384) is used to do multiplication along the 1600 elements.

So that's the bug, and you are welcome to add a proper broadcasting that solves your problem, or restructure your network to bypass the situation.  I don't remember whether GPU supports EXPAND_DIMS, but one way would be to expand (1, 1600, 1) to (1, 1600, 384) and then maybe multiply with a tensor with an explicit shape of (1, 1, 384).

tl;dr Broadcast in GPU delegate is broken; don't rely on it.  Be more explicit. Hi @impjdi thanks for the reply. 

Even I thought so 1,1600, 1 and 384 is not well formed but the logic is implemented in tensorflow core op and as well as in tensorflowlite-cpu op. It broadcasts each element in 2nd dim of first input and elements in 384 (i.e 1 1600, 1 and 1 1600, 384), to result in [1,1600, 384]. So wanted a clear idea to why there is limitation in GPU implementation for better clarity while writing a model.

Regarding the broadcasting error, I even interchanged both the inputs, still the error occurs for the case with 1d input, I feel there is limitation during GPU kernels or GPU broadcasting ( I couldn't find its code through grep, so cant comment on that).

for the case of 
model_3 : (1, 10, 1) x (1, 10, 3) => accuracy issue occurs. (mean absolute error (mae) = 2.44e-01).
we can just update this statement https://github.com/tensorflow/tensorflow/issues/60043#issuecomment-1476327924, it resolves the issue. (change the == comparison along channel dims to <=)

I tried to do include broadcast_to layer to prevent this, but this results in pack operation in tflite which does not have gpu implementation in r2.8.

Looks like the only way is to do manual broadcasting, but this layer is introduced when trying to do keras.layerNormalization on 3d input and converting this to Tflite. So manual broadcasting seems difficult  here, hence some help is needed.


Also, Is there any documentation or study material regarding tensorflowlite convertor or Tflite architecture in general, would like to do opensource contribution in the future. It would be tremendously helpful in understanding the codebase.
**System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22.04
- TensorFlow installed from (source or binary): source
- TensorFlow version (or github SHA if from source): r2.8
- TfliteGpuDelegate backend: cl

**Standalone code to reproduce the issue** 

While running the multiplication layer on TfliteGpuDelegate with input dimensions as {input1 - [1, 1600, 1] and input2 - [384]}, I am getting accuracy issues. The accuracy is fine while running on CPU. I am assuming there are limitations or issues in GPU execution during broadcasting. I have run experimentation on the following shapes and these are the observations:

[accuracy when compared the tflite cpu execution.]
model_1 : (1, 10, 3) x (1, 10, 3) => accuracy is fine.
model_2 : (1, 10, 3) x (1, 10, 1) => accuracy is fine.
model_3 : (1, 10, 1) x (1, 10, 3) => accuracy issue occurs. (mean absolute error (mae) = 2.44e-01).
model_4 : (1, 10, 1) x (3) => accuracy issue occurs. (mae = 2.71e-01).

PFA links for the models and detailed accuracy report is below.

models: https://drive.google.com/drive/folders/1DCfV1xPxdliJ7jtYF6LzhDjxTRM0uDSm?usp=share_link
accuracy report : https://drive.google.com/file/d/1wjLa1-KrgeyAzimEvhpq_ifRSfEOYeJr/view?usp=share_link

It would be helpful if the limitations are explained or the issues is debugged regarding this. Also any workaround for fixing (1, 1600, 1) x 384 will be appreciated. 

**Any other info / logs**

Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached.
