Call TensorFlowLite model with `CVPixelBuffer` from Camera
Hi, @mrousavy,

>1. The Frame is any arbitrary size, but the models are trained to specific sizes. So I'm obviously getting the following error:
>
>```Input tensor at index (9142529056) expects data size (110592), but got (8355840).```
>
>Input tensor at index (9142529056) expects data size (110592), but got (8355840). I want to avoid Frame resizing here and ideally have the TFLInterpreter accept the Frame (CMSampleBuffer) as is and do a stride/offset/jumps internally - is that even possible? If not, how can I figure out what Frame size I need to downscale to?

Currently we do not support dynamic input shapes, see: https://www.tensorflow.org/lite/guide/inference#run_inference_with_dynamic_shape_model

Unfortunately, we do not plan to support it soon due to performance considerations. Typically this means the input needs to be preprocessed into a static shape.

>2. The TFLInterpreter can only be invoked with NSData, and my Frame is a CMSampleBuffer allocated on the GPU. Is there any way to avoid this GPU -> CPU copy and stay on the GPU buffer the whole time? I have safe read access to that buffer in this callback.

I believe it can stay on the GPU Buffer, can you review this link and let me know if it answers your question? https://www.tensorflow.org/lite/ios/delegates/gpu#inputoutput_buffers_using_c_api

If this resolves your questions, please feel free to close the issue. Thanks for your reply @pkgoogle - I'm trying to use the C/C++ API instead of the Objective-C API

> Note: The following technique is only available when you are using Bazel or building TensorFlow Lite yourself. C++ API can't be used with CocoaPods.

There is a `TensorFlowLiteC` pod on CocoaPods, is that just shipping a pre-built XCFramework? Am I expected to just use that? I'm in an environment where using bazel is really hard to set up and distribute. Hi @mrousavy, TensorFlowLiteC is 
```
An internal-only pod containing the TensorFlow Lite C library that the public
  `TensorFlowLiteSwift` and `TensorFlowLiteObjC` pods depend on. This pod is not
  intended to be used directly. Swift developers should use the
  `TensorFlowLiteSwift` pod and Objective-C developers should use the
  `TensorFlowLiteObjC` pod.
```
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/ios/TensorFlowLiteC.podspec#11

So, if you're using the C/C++ API, then you have to use bazel/build from source to use the GPU API Gotcha, thanks. I'll stick to the Objective-C APIs then since using bazel is not really possible in my environment. No worries, @mrousavy, if you have no more open items for this issue please feel free to close.
Hey! Sorry I know this is off-topic and a question, but I couldn't find any examples or documentation on optimizing input-tensors for realtime usage.

I'm the author of a very popular Camera library for mobile apps (React Native / [VisionCamera](https://github.com/mrousavy/react-native-vision-camera)) and I'm trying to add a TensorFlow Lite integration to the Camera.

This should be as generic as possible and will allow the user to drop in any `.tflite` model which just gets called with the Camera frame (`CMSampleBuffer` on iOS, `android.media.Image` on Android) and returns _any_ data (output tensors).

I started with implementing the iOS part in Objective-C and set up my TensorFlow code like this:

```objc
NSString* modelPath = [[NSBundle mainBundle] pathForResource:@"model"
                                                      ofType:@"tflite"];
NSError* error;
TFLInterpreter* interpreter = [[TFLInterpreter alloc] initWithModelPath:modelPath
                                                                  error:&error];
if (error != nil) { /** ... */ }

[interpreter allocateTensorsWithError:&error];
if (error != nil) { /** ... */ }
```

And then I have my Camera Frame Callback which gets called for every Frame the Camera "sees" (60 times a second at 60 FPS):

```objc
auto imageBuffer = CMSampleBufferGetImageBuffer(frame.buffer);
auto bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
auto height = CVPixelBufferGetHeight(imageBuffer);
auto sourceBuffer = CVPixelBufferGetBaseAddress(imageBuffer);
auto inputData = [NSData dataWithBytesNoCopy:sourceBuffer length:bytesPerRow * height];

NSError* error;

TFLTensor *inputTensor = [interpreter inputTensorAtIndex:0 error:&error];
if (error != nil) { /** ... */ }

[inputTensor copyData:inputData error:&error];
if (error != nil) { /** ... */ }

[interpreter invokeWithError:&error];
if (error != nil) { /** ... */ }

TFLTensor *outputTensor = [interpreter outputTensorAtIndex:0 error:&error];
if (error != nil) { /** ... */ }

NSData *outputData = [outputTensor dataWithError:&error];
if (error != nil) { /** ... */ }

// TODO: return output data to user
```

Now I have two problems:

1. The Frame is any arbitrary size, but the models are trained to specific sizes. So I'm obviously getting the following error:

    ```
    Input tensor at index (9142529056) expects data size (110592), but got (8355840).
    ```

    I want to avoid Frame resizing here and ideally have the `TFLInterpreter` accept the Frame (`CMSampleBuffer`) _as is_ and do a stride/offset/jumps internally - is that even possible? If not, how can I figure out what Frame size I need to downscale to?
2. The `TFLInterpreter` can only be invoked with `NSData`, and my Frame is a `CMSampleBuffer` allocated on the GPU. Is there any way to avoid this GPU -> CPU copy and stay on the GPU buffer the whole time? I have safe read access to that buffer in this callback.

I've seen some MLKit samples (e.g. [MLKit Object Detection iOS](https://developers.google.com/ml-kit/vision/object-detection/ios)) and they allow you to just pass the `CMSampleBuffer` to the model - I'm wondering how this will be handled internally as it seems like the library is quite performant.

Also I found [this code on GitHub](https://github.com/kunass2/naildetector/blob/7ebfb2a3b7eaf6142be3d0a85e870bad4ce29d05/DeeplabModel.mm#L121-L205) which seemed quite interesting, but I couldn't figure out where `ProcessInputWithFloatModel(..)` is from...

As you can probably tell, I'm not an expert in this field but I'd appreciate any help here. Thanks!