Tflite: C++ API format to add NNAPI delegate
Hi @suyash-narain 

The NNAPI delegate is available with the TensorFlow Lite Interpreter in Java and Kotlin. 

For Android C APIs, please refer to [Android Native Developer Kit documentation](https://developer.android.com/ndk/guides/neuralnetworks).

Can you check [NAAPI support library](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/nnapi/sl#nnapi-support-library) and let us know if it helps?

Thanks. Hi @pjpratik 
I do not understand. You say NNAPI delegate is available in Java and Kotlin, but i have been able to use the external NNAPI delegate created with python API too. 
i have a nnapi_delegate.so file and i can load it to python script and execute. 
doesn't C++ have anything similar?

My C++ script already has these two headers:

#include "tensorflow/lite/tools/delegates/delegate_provider.h"
#include "tensorflow/lite/delegates/nnapi/nnapi_delegate.h"

And the label_image.cc makes use of NNAPI but i can't understand how or where is the NNAPI object is being created which is calling the delegate library .so file to execute. Some assistance here would be nice.

thanks

 also, @pjpratik @pkgoogle  if i have a delegate shared library file .so (say for NNAPI or any custom delegate), is there any way I can use that shared library when calling this delegate?
i know i can use this for hexagon tflite delegate, but does NNAPI or custom delegate tflite C++ api have it too? Hi @suyash-narain,

I was able to compile the minimal C++ example with your code modified like this:

```C++
/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <cstdio>

#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/optional_debug_tools.h"
#include "tensorflow/lite/delegates/nnapi/nnapi_delegate.h"

// This is an example that is minimal to read a model
// from disk and perform inference. There is no data being loaded
// that is up to you to add as a user.
//
// NOTE: Do not add any dependencies to this that cannot be built with
// the minimal makefile. This example must remain trivial to build with
// the minimal build tool.
//
// Usage: minimal <tflite model>

#define TFLITE_MINIMAL_CHECK(x)                              \
  if (!(x)) {                                                \
    fprintf(stderr, "Error at %s:%d\n", __FILE__, __LINE__); \
    exit(1);                                                 \
  }

int main(int argc, char* argv[]) {
  if (argc != 2) {
    fprintf(stderr, "minimal <tflite model>\n");
    return 1;
  }
  const char* filename = argv[1];

  // Load model
  std::unique_ptr<tflite::FlatBufferModel> model =
      tflite::FlatBufferModel::BuildFromFile(filename);
  TFLITE_MINIMAL_CHECK(model != nullptr);

  // Build the interpreter with the InterpreterBuilder.
  // Note: all Interpreters should be built with the InterpreterBuilder,
  // which allocates memory for the Interpreter and does various set up
  // tasks so that the Interpreter can read the provided model.
  tflite::ops::builtin::BuiltinOpResolver resolver;
  tflite::InterpreterBuilder builder(*model, resolver);
  std::unique_ptr<tflite::Interpreter> interpreter;
  builder(&interpreter);
  TFLITE_MINIMAL_CHECK(interpreter != nullptr);

  std::map<std::string, tflite::Interpreter::TfLiteDelegatePtr> delegates;
  auto delegate = tflite::Interpreter::TfLiteDelegatePtr(tflite::NnApiDelegate(), [](TfLiteDelegate*) {});
  delegates.emplace("NNAPI", std::move(delegate));
  for (const auto& delegate : delegates) {
      interpreter->ModifyGraphWithDelegate(delegate.second.get());
  }

  // Allocate tensor buffers.
  TFLITE_MINIMAL_CHECK(interpreter->AllocateTensors() == kTfLiteOk);
  printf("=== Pre-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Fill input buffers
  // TODO(user): Insert code to fill input tensors.
  // Note: The buffer of the input tensor with index `i` of type T can
  // be accessed with `T* input = interpreter->typed_input_tensor<T>(i);`

  // Run inference
  TFLITE_MINIMAL_CHECK(interpreter->Invoke() == kTfLiteOk);
  printf("\n\n=== Post-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Read output buffers
  // TODO(user): Insert getting data out code.
  // Note: The buffer of the output tensor with index `i` of type T can
  // be accessed with `T* output = interpreter->typed_output_tensor<T>(i);`

  return 0;
}
```

This tells me you probably have a compilation issue rather than anything else... Do you have the header in the right place? Are you using cmake/bazel to compile? Can you check how your build system if any looks for include headers? I am trying to compile it for an aarch64 linux system, using g++
I have all the libraries i need already installed in place and have all the right headers too, in my /usr/include

compilation command on target:

'g++ image_classification.cpp -o image_classification -lnnapi-support -lneuralnetworks -l/usr/lib/nnapi_delegate.so -ltensorflowlite -lopencv_objdetect -lopencv_features2d -lopencv_imgproc -lopencv_highgui -lopencv_core -lopencv_imgcodecs'

Can you please tell me your compilation command? I already have tf compiled and installed so building tf again doesn't make sense. 

I have a .so file which is used for NNAPI delegate. How do i include that .so file here? the library is nnapi_delegate.so
in python, i can simply provide it to 'experimental_delegates' argument. Anyway i can link it in C++ too?

thanks
 Hi @suyash-narain, I used the minimal example and cmake: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/examples/minimal, so cmake constructs the compiler commands for me. In general it is recommended that you use/learn a build system (cmake/bazel would be good choices) as that will automate some of these things for you. However to answer your question more directly, you need the -I flag to point to the Include source directory, and -L to link your .so file, there are probably multiple ways to accomplish this with environment variables and other g++ settings.

Please review these links for examples:
https://stackoverflow.com/questions/12654013/how-to-make-g-search-for-header-files-in-a-specific-directory
https://stackoverflow.com/questions/27208932/link-so-file-to-cpp-file-via-g-compiling This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61419">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61419">No</a>


### System information

-   **Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow)**: No
-   **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 20.04
-   **TensorFlow installed from (source or binary)**: binary
-   **TensorFlow version (use command below)**: 2.10
-   **Python version**: 3.10


### Describe the problem
I want to know what is the python API equivalent of adding a tflite delegate to execute the model. 
In python, we can directly add the argument 'experimental_delegates' to tflite.Interpreter and provide the path to the delegate .so file. 

If i want to add NNAPI or GPU delegate when using C++ API, what is the command for that? I couldn't find effective documentation to enable a delegate when using C++.

ModifyGraphWithDelegate is used, but how do i define a delegate up here? I want to make use of NNAPI delegate and i have the .so file for the same as well. Below is the code snippet am using to enable NNAPI delegate

```
  std::map<std::string, tflite::Interpreter::TfLiteDelegatePtr> delegates;
    auto delegate = tflite::Interpreter::TfLiteDelegatePtr(tflite::NnApiDelegate(), [](TfLiteDelegate*) {});
    delegates.emplace("NNAPI", std::move(delegate));
    for (const auto& delegate : delegates) {
        interpreter->ModifyGraphWithDelegate(delegate.second.get());
    } 
```

but when i compile the code i get the error:

>  undefined reference to tflite::NnApiDelegate()'

How can i enable NNAPI delegate with C++?

thanks


