TFLite GPU: Remove all 16-component vectors
Note: fixed a small build issue Can you provide some comprehensive benchmark values for FC & Conv Transpose? These are very important ops, and all changes to them must be done carefully  Hi @looi Can you please check @grantjensen's [comments](https://github.com/tensorflow/tensorflow/pull/62390#issuecomment-1819812742) and keep us posted ? Thank you! This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you. Sorry, i will post an update soon Hi @looi Any update on this PR? Please. Thank you! I've tested this change on the model [ssd_mobiledet_cpu_coco](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md) (without TFLite_Detection_PostProcess). This model contains FC and Conv Transpose ops. It is originally intended for running on CPU, but it runs fine using the tflite GPU plugin. For the purpose of testing, I also overrode `UseBufferForWeights` in [fully_connected.cc](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/common/tasks/fully_connected.cc) and [convolution_transposed.cc](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/common/tasks/convolution_transposed.cc) to return true. This change doesn't actually affect any device where that function returns false.

The performance is the same or slightly better on these devices:

* Pixel 3 (Adreno 630): 30ms -> 30ms
* Pixel 7a (Mali-G710): 25ms -> 24ms
* Intel UHD Graphics 630: 10ms -> 10ms

Basically, this change replaces memory accesses with 16-component vectors (float16) with accesses using 4-component vectors. I expect this change will produce a small but positive effect in performance, because [clpeak](https://github.com/krrishnarraj/clpeak) shows that memory bandwidth for float4 is generally same or better than float16 on mobile platforms. This is probably because float16 is not commonly used (nor is it supported by openGL/vulkan/directx/metal), and drivers are not optimized for it. For example, these are clpeak results:

Pixel 3 (Adreno 630)
```
Platform: QUALCOMM Snapdragon(TM)
  Device: QUALCOMM Adreno(TM)
    Driver version  : OpenCL 2.0 QUALCOMM build: commit #781e7d0 changeid #I46ff5fc46f Date: 12/01/20 Tue Local Branch: QPR1 Remote Branch:  Compiler E031.37.10.00 (Android)
    Compute units   : 2
    Clock frequency : 1 MHz

    Global memory bandwidth (GBPS)
      float   : 22.20
      float2  : 23.42
      float4  : 24.34
      float8  : 22.94
      float16 : 21.73
```

Pixel 7a (Mali-G710)
```
Platform: ARM Platform
  Device: Mali-G710 r0p0
    Driver version  : 3.0 (Android)
    Compute units   : 7
    Clock frequency : 5 MHz

    Global memory bandwidth (GBPS)
      float   : 36.29
      float2  : 34.98
      float4  : 34.59
      float8  : 33.64
      float16 : 19.63
```

Intel UHD Graphics 630
```
Platform: Intel(R) OpenCL HD Graphics
  Device: Intel(R) UHD Graphics [0x9bc4]
    Driver version  : 1.0.0 (Linux x64)
    Compute units   : 24
    Clock frequency : 1150 MHz

    Global memory bandwidth (GBPS)
      float   : 20.61
      float2  : 23.38
      float4  : 22.70
      float8  : 23.89
      float16 : 23.55
```

Please let me know if this is helpful, or if you desire other information. Thanks! Hi @grantjensen Can you please review this PR ? Thank you! Yeah, overall I like this change. This is a nice little find :). Speed performance values look good. The one remaining concern I have is the memory performance. I noticed the Pixel 7a values you provided float4 & float16 differ dramatically. Could you see what the memory looks like for benchmarking a model w & w/o your change on a Pixel 7a? For the model above, I observed a small improvement on Pixel 7a (25ms -> 24ms). Compared to Pixel 3 and Intel UHD 630, this is the only device where I observed a consistent (but small) improvement, probably because of the float16 performance difference shown by clpeak. Is there another kind of benchmark you are looking for? Nah that looks good to me. Last question though, why remove ReplaceAllWords as well?  > Nah that looks good to me. Last question though, why remove ReplaceAllWords as well?

The function in that file appears to be unused now. The same or similar function called ReplaceAllWords seems to be duplicated in other files as well, like https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/metal/metal_arguments.cc
This simplifies the code, removes the text replacement hack for metal, and probably provides the same or better performance on all platforms. It avoids the need for similar hacks in other platforms too, because 16-component vectors are not commonly supported.