tflite-rutime: RuntimeError: Encountered unresolved custom op: FarthestPointSample.
Hi @kuangzy2011 

The tflite-runtime has a fraction the size of the full tensorflow package and includes the bare minimum code required to run inferences. 

To resolve the custom operators, we need to define own custom implementation of an unsupported TensorFlow operator in TensorFlow Lite.

We need to [create and register the operator](https://www.tensorflow.org/lite/guide/ops_custom#create_and_register_the_operator) so that the TensorFlow Lite runtime knows how to map your operator and parameters in your graph to executable C/C++ code.

Please refer to this [documentation](https://www.tensorflow.org/lite/guide/ops_custom#example_custom_atan_operator) on examples for resolving custom ops.

Thanks. REGISTER_OP("FarthestPointSample")
  .Attr("npoint: int")
  .Input("inp: float32")
  .Output("out: int32")

c->GetAttr("npoint", &npoint)


This custom OP is in tensorflow/core/kernels, it has Attr, but how to get this attr in tensorflow/lite/kernels OP? Hi @kuangzy2011 

Can you provide more details about your issue?

We have to create a custom `FarthestPointSample.cc` in tensorflow/lite/kernels and register in 

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/custom_ops_register.h

Please check `Atan2` op for reference asn provided in the documentation.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/atan2_custom.cc

Thanks. @pjpratik 
    In pointnet++ project, compiles following custom OP to a share library, and load it with tf.load_op_library to train the model.
After the training is done, convert this model to tflite with "converter.allow_custom_ops = True".
    When tried to do inference with tflite-runtime, it raised error "RuntimeError: Encountered unresolved custom op: FarthestPointSample."
    I noticed the OP in tensorflow/core/kernels is different with that in tensorflow/lite/kernels, so I am trying to convert these OPs to fit tensorflow/lite/kernels. There is Attr in following OP, but not found any related information in  tensorflow/lite/kernels OP.


   Source target:
https://github.com/charlesq34/pointnet2/blob/master/tf_ops/sampling/tf_sampling.cpp
https://github.com/charlesq34/pointnet2/blob/master/tf_ops/sampling/tf_sampling_g.cu
```
/* Furthest point sampling
 * Original author: Haoqiang Fan
 * Modified by Charles R. Qi
 * All Rights Reserved. 2017.
 */
#include "tensorflow/core/framework/op.h"
#include "tensorflow/core/framework/op_kernel.h"
#include "tensorflow/core/framework/shape_inference.h"
#include "tensorflow/core/framework/common_shape_fns.h"
#include <cuda_runtime.h>

using namespace tensorflow;

REGISTER_OP("ProbSample")
  .Input("inp: float32")
  .Input("inpr: float32")
  .Output("out: int32")
  .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
    ::tensorflow::shape_inference::ShapeHandle dims1; // batch_size * ncategory
    TF_RETURN_IF_ERROR(c->WithRank(c->input(0), 2, &dims1));
    ::tensorflow::shape_inference::ShapeHandle dims2; // batch_size * npoints
    TF_RETURN_IF_ERROR(c->WithRank(c->input(1), 2, &dims2));
    // batch_size * npoints
    ::tensorflow::shape_inference::ShapeHandle output = c->MakeShape({c->Dim(dims2, 0), c->Dim(dims2, 1)});
    c->set_output(0, output);
    return OkStatus();
  });
REGISTER_OP("FarthestPointSample")
  .Attr("npoint: int")                                                      >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Attr
  .Input("inp: float32")
  .Output("out: int32")
  .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
    ::tensorflow::shape_inference::ShapeHandle dims1; // batch_size * npoint * 3
    TF_RETURN_IF_ERROR(c->WithRank(c->input(0), 3, &dims1));
    int npoint;
    TF_RETURN_IF_ERROR(c->GetAttr("npoint", &npoint));
    ::tensorflow::shape_inference::ShapeHandle output = c->MakeShape({c->Dim(dims1, 0), npoint});
    c->set_output(0, output);
    return OkStatus();
  });
REGISTER_OP("GatherPoint")
  .Input("inp: float32")
  .Input("idx: int32")
  .Output("out: float32")
  .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
    ::tensorflow::shape_inference::ShapeHandle dims1; // batch_size * ndataset * 3
    TF_RETURN_IF_ERROR(c->WithRank(c->input(0), 3, &dims1));
    ::tensorflow::shape_inference::ShapeHandle dims2; // batch_size * npoints
    TF_RETURN_IF_ERROR(c->WithRank(c->input(1), 2, &dims2));
    // batch_size * npoints * 3
    ::tensorflow::shape_inference::ShapeHandle output = c->MakeShape({c->Dim(dims1, 0), c->Dim(dims2, 1), c->Dim(dims1, 2)});
    c->set_output(0, output);
    return OkStatus();
  });
REGISTER_OP("GatherPointGrad")
  .Input("inp: float32")
  .Input("idx: int32")
  .Input("out_g: float32")
  .Output("inp_g: float32")
  .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
    c->set_output(0, c->input(0));
    return OkStatus();
  });

void probsampleLauncher(int b,int n,int m,const float * inp_p,const float * inp_r,float * temp,int * out);
class ProbSampleGpuOp: public OpKernel{
  public:
    explicit ProbSampleGpuOp(OpKernelConstruction* context):OpKernel(context){}
    void Compute(OpKernelContext * context)override{
      const Tensor& inp_tensor=context->input(0);
      const Tensor& inpr_tensor=context->input(1);
      auto inp_flat=inp_tensor.flat<float>();
      auto inpr_flat=inpr_tensor.flat<float>();
      const float * inp=&(inp_flat(0));
      const float * inpr=&(inpr_flat(0));
      OP_REQUIRES(context,inp_tensor.dims()==2,errors::InvalidArgument("ProbSample expects (batch_size,num_choices) inp shape"));
      int b=inp_tensor.shape().dim_size(0);
      int n=inp_tensor.shape().dim_size(1);
      OP_REQUIRES(context,inpr_tensor.dims()==2 && inpr_tensor.shape().dim_size(0)==b,errors::InvalidArgument("ProbSample expects (batch_size,num_points) inpr shape"));
      int m=inpr_tensor.shape().dim_size(1);
      Tensor * out_tensor=NULL;
      OP_REQUIRES_OK(context,context->allocate_output(0,TensorShape{b,m},&out_tensor));
      auto out_flat=out_tensor->flat<int>();
      int * out=&(out_flat(0));
      Tensor temp_tensor;
      OP_REQUIRES_OK(context,context->allocate_temp(DataTypeToEnum<float>::value,TensorShape{b,n},&temp_tensor));
      auto temp_flat=temp_tensor.flat<float>();
      float * temp=&(temp_flat(0));
      probsampleLauncher(b,n,m,inp,inpr,temp,out);
    }
};
REGISTER_KERNEL_BUILDER(Name("ProbSample").Device(DEVICE_GPU), ProbSampleGpuOp);

void farthestpointsamplingLauncher(int b,int n,int m,const float * inp,float * temp,int * out);
class FarthestPointSampleGpuOp: public OpKernel{
  public:
    explicit FarthestPointSampleGpuOp(OpKernelConstruction* context):OpKernel(context) {
### >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>GetAttr
                    OP_REQUIRES_OK(context, context->GetAttr("npoint", &npoint_));    
                    OP_REQUIRES(context, npoint_ > 0, errors::InvalidArgument("FarthestPointSample expects positive npoint"));
                }
    void Compute(OpKernelContext * context)override{
      int m = npoint_;

      const Tensor& inp_tensor=context->input(0);
      OP_REQUIRES(context,inp_tensor.dims()==3 && inp_tensor.shape().dim_size(2)==3,errors::InvalidArgument("FarthestPointSample expects (batch_size,num_points,3) inp shape"));
      int b=inp_tensor.shape().dim_size(0);
      int n=inp_tensor.shape().dim_size(1);
      auto inp_flat=inp_tensor.flat<float>();
      const float * inp=&(inp_flat(0));
      Tensor * out_tensor;
      OP_REQUIRES_OK(context,context->allocate_output(0,TensorShape{b,m},&out_tensor));
      auto out_flat=out_tensor->flat<int>();
      int * out=&(out_flat(0));
      Tensor temp_tensor;
      OP_REQUIRES_OK(context,context->allocate_temp(DataTypeToEnum<float>::value,TensorShape{32,n},&temp_tensor));
      auto temp_flat=temp_tensor.flat<float>();
      float * temp=&(temp_flat(0));
      farthestpointsamplingLauncher(b,n,m,inp,temp,out);
    }
    private:
        int npoint_;
};
REGISTER_KERNEL_BUILDER(Name("FarthestPointSample").Device(DEVICE_GPU),FarthestPointSampleGpuOp);

void gatherpointLauncher(int b,int n,int m,const float * inp,const int * idx,float * out);
class GatherPointGpuOp: public OpKernel{
  public:
    explicit GatherPointGpuOp(OpKernelConstruction * context):OpKernel(context){}
    void Compute(OpKernelContext * context)override{
      const Tensor& inp_tensor=context->input(0);
      OP_REQUIRES(context,inp_tensor.dims()==3 && inp_tensor.shape().dim_size(2)==3,errors::InvalidArgument("GatherPoint expects (batch_size,num_points,3) inp shape"));
      int b=inp_tensor.shape().dim_size(0);
      int n=inp_tensor.shape().dim_size(1);
      const Tensor& idx_tensor=context->input(1);
      OP_REQUIRES(context,idx_tensor.dims()==2 && idx_tensor.shape().dim_size(0)==b,errors::InvalidArgument("GatherPoint expects (batch_size,num_result) idx shape"));
      int m=idx_tensor.shape().dim_size(1);
      auto inp_flat=inp_tensor.flat<float>();
      const float * inp=&(inp_flat(0));
      auto idx_flat=idx_tensor.flat<int>();
      const int * idx=&(idx_flat(0));
      Tensor * out_tensor=NULL;
      OP_REQUIRES_OK(context,context->allocate_output(0,TensorShape{b,m,3},&out_tensor));
      auto out_flat=out_tensor->flat<float>();
      float * out=&(out_flat(0));
      gatherpointLauncher(b,n,m,inp,idx,out);
    }
};
REGISTER_KERNEL_BUILDER(Name("GatherPoint").Device(DEVICE_GPU),GatherPointGpuOp);

void scatteraddpointLauncher(int b,int n,int m,const float * out_g,const int * idx,float * inp_g);
class GatherPointGradGpuOp: public OpKernel{
  public:
    explicit GatherPointGradGpuOp(OpKernelConstruction * context):OpKernel(context){}
    void Compute(OpKernelContext * context)override{
      const Tensor& inp_tensor=context->input(0);
      OP_REQUIRES(context,inp_tensor.dims()==3 && inp_tensor.shape().dim_size(2)==3,errors::InvalidArgument("GatherPointGradGpuOp expects (batch_size,num_points,3) inp"));
      int b=inp_tensor.shape().dim_size(0);
      int n=inp_tensor.shape().dim_size(1);
      const Tensor& idx_tensor=context->input(1);
      OP_REQUIRES(context,idx_tensor.dims()==2 && idx_tensor.shape().dim_size(0)==b,errors::InvalidArgument("GatherPointGradGpuOp expects (batch_size,num_result) idx shape"));
      int m=idx_tensor.shape().dim_size(1);
      auto inp_flat=inp_tensor.flat<float>();
      const float * inp=&(inp_flat(0));
      auto idx_flat=idx_tensor.flat<int>();
      const int * idx=&(idx_flat(0));
      const Tensor& out_g_tensor=context->input(2);
      OP_REQUIRES(context,out_g_tensor.dims()==3 && out_g_tensor.shape().dim_size(0)==b && out_g_tensor.shape().dim_size(1)==m && out_g_tensor.shape().dim_size(2)==3,errors::InvalidArgument("GatherPointGradGpuOp expects (batch_size,num_result,3) out_g shape"));
      auto out_g_flat=out_g_tensor.flat<float>();
      const float * out_g=&(out_g_flat(0));
      Tensor * inp_g_tensor=NULL;
      OP_REQUIRES_OK(context,context->allocate_output(0,TensorShape{b,n,3},&inp_g_tensor));
      auto inp_g_flat=inp_g_tensor->flat<float>();
      float * inp_g=&(inp_g_flat(0));
      cudaMemset(inp_g,0,b*n*3*4);
      scatteraddpointLauncher(b,n,m,out_g,idx,inp_g);
    }
};
REGISTER_KERNEL_BUILDER(Name("GatherPointGrad").Device(DEVICE_GPU),GatherPointGradGpuOp);
``` Hi @kuangzy2011 

Thanks for the information.

Tensorflow and TFLite APIs for creating op kernels are, at the moment, not identical. Check [tf-shim](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/kernels/shim) which creates a shim over the custom op APIs of TF and TFLite which might help in your case.

Thanks.

 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61521">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61521">No</a>
 @pjpratik ,

    I added shim APIs, but how to compile them into the library.
    I rebuilt tflite_runtime and it raised the same errors.

    Here is the change.

https://github.com/kuangzy2011/tensorflow/blob/main/tensorflow/lite/kernels/shim/test_op/sampling_op.h
https://github.com/kuangzy2011/tensorflow/blob/main/tensorflow/lite/kernels/shim/test_op/sampling_tflite_op.cc
https://github.com/kuangzy2011/tensorflow/blob/main/tensorflow/lite/kernels/shim/test_op/sampling_tflite_op.h

https://github.com/kuangzy2011/tensorflow/blob/98a7af9c8383a1e1fde154700bcc66493277298e/tensorflow/lite/kernels/shim/test_op/BUILD#L156 Hi @kuangzy2011 

Could you please the share the error log and steps you have followed in order to better understand and reproduce the issue?

Thanks. > Hi @kuangzy2011
> 
> Could you please the share the error log and steps you have followed in order to better understand and reproduce the issue?
> 
> Thanks.

@pjpratik ,

    It works now. Need to add test_op to lite/kernels/BUILD
    Thanks.
 @kuangzy2011 

Glad it works. Please feel free to close the issue since it is resolved.

Thanks. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61521">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61521">No</a>
 If I understand this correctly - you would have to build `tflite-runtime` with the custom operator in order to use a tflite model that requires that custom operator? There's no way provide access to the custom operator via a shared library object that is provided at runtime?

Although - it does seem like there's some mechanism via `tflite_runtime.interpreter.InterpreterWithCustomOps`; but this isn't part of the public API.
**System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04
- TensorFlow installed from (source or binary): source 
- TensorFlow version (or github SHA if from source): 2.13.0


**Provide the text output from tflite_convert**
I did some test with pointnet++(https://github.com/charlesq34/pointnet2), and tried to inference with tf.lite or tflite-runtime, but both of them show the error message below:
```
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    predict = PointNetPredict('/kaggle/input/model-sign/model_sign.tflite')
  File "/kaggle/working/pointnet3c1/models/pointnet_predict.py", line 27, in __init__
    self.interpreter = self.init_model()
  File "/kaggle/working/pointnet3c1/models/pointnet_predict.py", line 39, in init_model
    interpreter.allocate_tensors()
  File "/opt/conda/lib/python3.7/site-packages/tensorflow/lite/python/interpreter.py", line 513, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Encountered unresolved custom op: FarthestPointSample.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom Node number 0 (FarthestPointSample) failed to prepare.Encountered unresolved custom op: FarthestPointSample.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom Node number 0 (FarthestPointSample) failed to prepare.
```
I noticed there were some custom ops(tf_ops) in project pointnet2, but how to convert these ops to tflite-runtime operators?

```
# Copy and paste here
```

**Standalone code to reproduce the issue** 
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.

Also, please include a link to a GraphDef or the model if possible.

**Any other info / logs**

Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached.
