Not able to get TFlite Model Maker Object Detector to produce expected output
Hi @jansdhillon 

The output shapes for the tflite model can be observed using `interpreter.get_output_details()`. The outputs might be ordered and need to be accessed accordingly. 

Could you please share a toy tflite model in order to debug the issue?

Thanks. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. @jansdhillon Were you able to solve this and could please post the solution to this issue? I am having the same problem at the moment > Hi @jansdhillon
> 
> The output shapes for the tflite model can be observed using `interpreter.get_output_details()`. The outputs might be ordered and need to be accessed accordingly.
> 
> Could you please share a toy tflite model in order to debug the issue?
> 
> Thanks.

I did run that, which you can see the output of in the last block of code. They all have a shape of []. > @jansdhillon Were you able to solve this and could please post the solution to this issue? I am having the same problem at the moment

Unfortunately, no I was not. I was trying to use this for a project but wasn't able to get it working in time so I had to pivot. Alright thanks for responding I will try to reproduce! I have run model maker on two different datasets. With one the inference afer conversion works just fine. With the other (the current one) I got the same results as you. Maybe I can find out the reason I was not able to reproduce it again. I am also using WSL with Ubuntu 22.04. I tested the same model as you with model maker and this time loading worked. However, when exporting the model to Tflite I also got this

 `2023-11-13 00:52:04.885225: W tensorflow/core/common_runtime/graph_constructor.cc:803] Node 'resample_p7/PartitionedCall' has 1 outputs but the _output_shapes attribute specifies shapes for 3 outputs. Output shapes may be inaccurate.
`

But it did not matter. Loading the model and inferring it worked fine afterwards. I was able to access to output tensors like this:

```
  signature_fn = interpreter.get_signature_runner()
  output = signature_fn(images=image)

  count = int(np.squeeze(output['output_0']))
  scores = np.squeeze(output['output_1'])
  classes = np.squeeze(output['output_2'])
  boxes = np.squeeze(output['output_3'])

```

 Hi @jansdhillon 

As mentioned @alexw92 , use the signature function to get the output details. You can add the following code to the interpreter. This information can be found in the [documentation](https://www.tensorflow.org/lite/models/modify/model_maker/object_detection ) as well. 



```
def detect_objects(interpreter, image, threshold):
  """Returns a list of detection results, each a dictionary of object info."""

  signature_fn = interpreter.get_signature_runner()

  # Feed the input image to the model
  output = signature_fn(images=image)

  # Get all outputs from the model
  count = int(np.squeeze(output['output_0']))
  scores = np.squeeze(output['output_1'])
  classes = np.squeeze(output['output_2'])
  boxes = np.squeeze(output['output_3'])

  results = []
  for i in range(count):
    if scores[i] >= threshold:
      result = {
        'bounding_box': boxes[i],
        'class_id': classes[i],
        'score': scores[i]
      }
      results.append(result)
  return results
```
Please let us know if any problem persists.

Thank You This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62135">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62135">No</a>
 @alexw92 Hi, I'm not sure if you've deleted your comment but I got a notification about it so I figured I'd respond.

That was the same issue I was having. I should have been more clear about what I was actually trying to do, which was use the tflite model directly in my Flutter app with tflite_flutter. However, as you mentioned, it seems like the only way to get the actual output is with Python, which obviously does not work with Flutter directly. 

Of course, you could use a Flask server but at that point what's the point of on-device inference, really. This is why I pivoted away from this approach.

But I do agree, it would be ideal if the TFLite Model Maker could create models that work outside with the Flutter package. I did have some better results using MediaPipe but ultimately scrapped Flutter entirely. @jansdhillon Hi, yes sorry I did delete it. I thought I could reopen the issue but it does not work :D

Yes I fully aggree! I had to do manual fixes which are not appropriate in flutter tflite package to make this work. We should not be forced to do it like that. And since both Tensorflow, Flutter and also now the Flutter TFLite Package are under the hood of Google there should also be some interest of Google to make these work together I guess. 

Until this is getting resolved if you or anyone else want to come back to use Flutter TFLite you can refer to this [issue](https://github.com/tensorflow/flutter-tflite/issues/67) which describes what must be done to in Flutter TFLite to use model maker OD models.
### 1. System information

- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): WSL Linux Ubuntu 20.04 (although I've also tried on Linux Ubuntu 22.04)
- TensorFlow installation (pip package or built from source): Pip package (t.
- TensorFlow library (version, if pip package or github SHA, if built from source):
- I built it with Conda and Python 3.9. Basically I install the environment in a Jupyter kernel and then use it to install the packages and eventually run the script.

```
# Install Miniconda
!apt install wget
!wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.9.2-Linux-x86_64.sh
!chmod +x Miniconda3-py37_4.9.2-Linux-x86_64.sh
!bash ./Miniconda3-py37_4.9.2-Linux-x86_64.sh -b -f -p /usr/local

# Update Conda
!conda update -n base -c defaults conda -y

# #Create a Python 3.9 environment
!conda create --name py39_environment python=3.9 -y

# # Initialize shell for Conda
!conda init bash

# # Activate the environment and check Python version
!source activate py39_environment && python --version
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip install tflite_model_maker
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip install -q pycocotools
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip install opencv-python-headless
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip uninstall -y tensorflow && pip install -q tensorflow==2.8.0
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip install ipykernel
!source /usr/local/etc/profile.d/conda.sh && conda activate py39_environment && pip install --upgrade numba llvmlite
```

### 2. Code

Provide code to help us reproduce your issues using one of the following options:

train.py:

```
import numpy as np
import os

np.object = object
np.bool = bool
np.complex = complex

from tflite_model_maker.config import ExportFormat
from tflite_model_maker import model_spec
from tflite_model_maker import object_detector

import tensorflow as tf
assert tf.__version__.startswith('2')

tf.get_logger().setLevel('ERROR')
from absl import logging
logging.set_verbosity(logging.ERROR)

label_map = {1: 'Plastic', 2: 'Trash'} 

train_images_dir = '/trashnet-training/pascal_voc/train/images'
train_annotations_dir = '/trashnet-training/pascal_voc/train/Annotations'
val_images_dir = '/trashnet-training/pascal_voc/valid/images/'
val_annotations_dir = '/trashnet-training/pascal_voc/valid/Annotations'
# test_images_dir =  '/trashnet-training/pascal_voc/train/Annotations'
# test_annotations_dir = '/trashnet-training/pascal_voc/train/Annotations'

train_data = object_detector.DataLoader.from_pascal_voc(
    train_images_dir, train_annotations_dir, label_map=label_map)

validation_data = object_detector.DataLoader.from_pascal_voc(
    val_images_dir, val_annotations_dir, label_map=label_map)

# test_data = object_detector.DataLoader.from_pascal_voc(
#     test_images_dir, test_annotations_dir, label_map=label_map)

spec = model_spec.get('efficientdet_lite0')

model = object_detector.create(train_data=train_data, 
                               model_spec=spec, 
                               validation_data=validation_data, 
                               epochs=50, 
                               batch_size=10, 
                               train_whole_model=True)

model.evaluate(validation_data)

TFLITE_FILENAME = 'trained_model.tflite'
LABELS_FILENAME = 'model-labels.txt'

model.export(export_dir='.', tflite_filename=TFLITE_FILENAME, label_filename=LABELS_FILENAME,
             export_format=[ExportFormat.TFLITE, ExportFormat.LABEL])
```

test_model.py:

```
import tensorflow as tf
import numpy as np


interpreter = tf.lite.Interpreter(model_path="trained_model.tflite")
interpreter.allocate_tensors()


input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
input_data = interpreter.get_tensor(input_details[0]['index'])
print(f"Input tensor: {input_data}")
print(f"Output tensor: {output_data}")


tf.lite.experimental.Analyzer.analyze(model_path="trained_model.tflite")

```

My dataset is in the Pascal VOC format with train and valid subfolders, each with their own images/ and Annotations/ folders. I've been able to train the model several times but whenever I put the resulting .tflite into the testing script, it doesn't product the shape it's supposed to. According to the TF documentation for the TFLite Object Detector export_tflite function [here](https://github.com/tensorflow/examples/blob/master/tensorflow_examples/lite/model_maker/core/task/model_spec/object_detector_spec.py#L348-L401) the output shape should be:

    Four Outputs:
      detection_boxes: a float32 tensor of shape [1, num_boxes, 4] with box
        locations.
      detection_classes: a float32 tensor of shape [1, num_boxes] with class
        indices.
      detection_scores: a float32 tensor of shape [1, num_boxes] with class
        scores.
      num_boxes: a float32 tensor of size 1 containing the number of detected
        boxes.

### 3. Failure after conversion
 The training seems to go fine, although somewhere along the way it warns me that:
`2023-10-17 04:41:28.736735: W tensorflow/core/common_runtime/graph_constructor.cc:803] Node 'resample_p7/PartitionedCall' has 1 outputs but the _output_shapes attribute specifies shapes for 3 outputs. Output shapes may be inaccurate.` 
Which is concerning. And I think part of the problem because when I run my testing script on the model the output tensors are all blank:

```
...
d_4;class_net/class-predict/bias11) shape:[1, 3, 3, 18], type:INT8
  T#583(Reshape) shape:[1, 14400, 2], type:INT8
  T#584(Reshape_1) shape:[1, 14400, 4], type:INT8
  T#585(Reshape_2) shape:[1, 3600, 2], type:INT8
  T#586(Reshape_3) shape:[1, 3600, 4], type:INT8
  T#587(Reshape_4) shape:[1, 900, 2], type:INT8
  T#588(Reshape_5) shape:[1, 900, 4], type:INT8
  T#589(Reshape_6) shape:[1, 225, 2], type:INT8
  T#590(Reshape_7) shape:[1, 225, 4], type:INT8
  T#591(Reshape_8) shape:[1, 81, 2], type:INT8
  T#592(concat) shape:[1, 19206, 2], type:INT8
  T#593(Sigmoid) shape:[1, 19206, 2], type:INT8
  T#594(Sigmoid1) shape:[1, 19206, 2], type:FLOAT32
  T#595(Reshape_9) shape:[1, 81, 4], type:INT8
  T#596(concat_1) shape:[1, 19206, 4], type:INT8
  T#597(tfl.dequantize) shape:[1, 19206, 4], type:FLOAT32
  T#598(StatefulPartitionedCall:3) shape:[], type:FLOAT32
  T#599(StatefulPartitionedCall:2) shape:[], type:FLOAT32
  T#600(StatefulPartitionedCall:1) shape:[], type:FLOAT32
  T#601(StatefulPartitionedCall:0) shape:[], type:FLOAT32

---------------------------------------------------------------
Your TFLite model has ‘1’ signature_def(s).

Signature#0 key: 'serving_default'
- Subgraph: Subgraph#0
- Inputs: 
    'images' : T#0
- Outputs: 
    'output_0' : T#601
    'output_1' : T#600
    'output_2' : T#599
    'output_3' : T#598

---------------------------------------------------------------
              Model size:    4444720 bytes
    Non-data buffer size:     799672 bytes (17.99 %)
  Total data buffer size:    3645048 bytes (82.01 %)
    (Zero value buffers):        256 bytes (00.01 %)
```

I've trained it several times with different formats but I just can't get the right output shapes, every time they're blank Could someone help me to understand why this is not working? Any guidance or insight would be greatly appreciated.
