Tensorflow Lite Model Maker model works on Python API but not on device (IOS/Android)
Hi @carl-krikorian,

Thanks for reporting this issue. Can you please follow the instructions to fill out the issue template: https://github.com/tensorflow/tensorflow/blob/master/ISSUE_TEMPLATE.md, this will help us identify what is different about your environment which may be causing the issue. Also can you clarify whether this is happening for the Swift API, Android API, or both? Also any additional real custom code you are executing will help as well. A minimal example to reproduce is preferred. Hello @pkgoogle,
To answer your question, the issue was encountered on both Swift and Android. For the Swift API, the app would show the error displayed in the first comment, whereas for the Android API the app seemed to crash. 


### Have I written custom code (as opposed to using a stock example script provided in TensorFlow):

For the Android app we used the sample app found [here](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier#step_2_run_inference_using_the_api). Similarly for Swift we followed these instructions [here](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier#step_2_run_inference_using_the_api_2).
Here is a sample of the struct used for the Swift code in the first image:
```

- (NSDictionary<NSString *, NSNumber *> *)classifyWithText:(NSString *)text {

  struct Categories *cCategories = BertNLClassifierClassify(_bertNLClassifier, text.UTF8String);

  NSMutableDictionary<NSString *, NSNumber *> *ret = [NSMutableDictionary dictionary];

  for (int i = 0; i < cCategories->size; i++) {

    struct Category cCategory = cCategories->categories[i];

    [ret setValue:[NSNumber numberWithDouble:cCategory.score]

           forKey:[NSString stringWithUTF8String:cCategory.text]];

  }

  NLClassifierCategoriesDelete(cCategories);

  return ret;

}
```
The Python inference that worked was run using this code block

```
import os
import pandas as pd
import contractions
import re
import nltk
import unicodedata
from tflite_model_maker.text_classifier import MobileBertClassifierSpec

def remove_special_characters(text):
    # define the pattern to keep
    pat = r'[^a-zA-z]' 
    text = re.sub(pat, ' ', text)
    text = re.sub(" +", " ", text)
    text = text.strip()
    return text

def preprocess(target_string):
    target_string = unicodedata.normalize('NFKD', target_string).encode('ascii', 'ignore').decode('utf-8', 'ignore')
    target_string = contractions.fix(target_string)
    target_string = remove_special_characters(target_string)    

    lemmatizer = WordNetLemmatizer()
    # Lemmatization
    words = nltk.word_tokenize(target_string)
    words = [lemmatizer.lemmatize(word) for word in words if word not in set(stopwords.words('english'))]
    target_string = ' '.join(words)
    target_string = target_string.lower()
    return target_string

cur_model = BertNLClassifier.create_from_file(tflite_model_path)

def make_prediction(cur_input, model):
    cur_input = preprocess(cur_input)
    res = model.classify(cur_input)
    classifications = res.classifications
    return max(classifications[0].categories, key=lambda x: x.score)

make_prediction(text, cur_model)
```
### OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
The model was trained on the fallback version of Google Colab, It is no longer available but I believe it used either Ubuntu 20.04 or 18.04.
The inference testing was made on Ubuntu 22.04 with the Python API and for the Android and Swift APIs on Mac OS 16.5.

### Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on a mobile device:
The issue was while running the app on emulators.

### TensorFlow installed from (source or binary):
The packages were install through pip on both Google Colab and the Ubuntu machine for training.

### TensorFlow version (use command below):
For the Python inference the version of Tensorflow was:
v2.8.1-10-g2ea19cbb575 2.8.2

### Bazel version
Not applicable

### Python version / GCC/Compiler version
Was trained on:
  3.9.16 (main, Dec  7 2022, 01:11:51) 
  [GCC 9.4.0]
Inference worked on Python API using:
  3.7.11 (default, Jul 27 2021, 14:32:16) 
  [GCC 7.5.0]

### CUDA/cuDNN version:
CUDA Version: 12.0

### GPU model and memory:
Tesla T4 16 GB of Memory


Let me know if I can provide any more specific or further information. Thank you for your time! Hi @carl-krikorian, can you upload the .tflite model you used that ran into this issue? The code you used to produce the model would also be helpful (the code sample you provided appears to create one and run inference but doesn't do the actual conversion). Hello @pkgoogle, 
The model was attached in the first comment here: https://drive.google.com/file/d/1jeKm7EesBZqi_lgPrCSq_HwPapX54OlL/view?usp=sharing
I would have zipped and sent it but the last time I did that it wiped the entire MedtaData.
Here is the training script I used: 
[tflite_maker.zip](https://github.com/tensorflow/tensorflow/files/11674325/tflite_maker.zip)
 Hi @carl-krikorian,

I am having trouble replicating the issue, let's focus on iOS/Swift for now.

How are you installing the pod? How are you calling the library? How are you including the "model.tflite" file in the app? Is it part of the app's bundle? Exact steps, exact podfile and a minimal reproducing code example is preferred (Are you calling it in the app's init function?) Are you using an emulator? If so what is your exact target? Are you targeting arm64 or x86_64?

Generally the more information and the clearer I can reproduce your environment the easier it will be for us to help you. Thanks for your help! Hello @pkgoogle 
- We are installing the pod using: 
  pod 'TensorFlowLiteSwift'
  pod 'TensorFlowLiteTaskText', '~> 0.2.0'
- This is we are calling the library:
  guard let modelPath = Bundle.main.path(forResource: "model", ofType: "tflite") else { return }
  self.tensorFlowBertNLClassifier = TFLBertNLClassifier.bertNLClassifier(modelPath: modelPath) 
- The model "model.tflite" is added in a folder: AppFolder -> Classification -> model.tflite
- The model "model.tflite" is included in the target.
- We are using a real device: iPhone XS with iOS 16.5

Thanks 


 Hi @carl-krikorian, and @pauljasser,

I was able to successfully build a minimal working example xcode app and tested it on an iPhone XS emulator with iOS **16.4**

Here's my podfile (test60743 is my toy project)
```ruby
# Uncomment the next line to define a global platform for your project
# platform :ios, '9.0'

target 'test60743' do
  # Comment the next line if you don't want to use dynamic frameworks
  use_frameworks!

  # Pods for test60743
  pod 'TensorFlowLiteTaskText', '~> 0.2.0'

end
```

Here's the main code of my toy app, everything else is default/unchanged:
```swift
//
//  test60743App.swift
//  test60743
//
//

import SwiftUI
import TensorFlowLiteTaskText

@main
struct test60743App: App {
    init() {
        guard let modelPath = Bundle.main.path(forResource: "model", ofType: "tflite") else { return }

        // Initialization
        let bertNLClassifier = TFLBertNLClassifier.bertNLClassifier(
              modelPath: modelPath)

        // Run inference
        let categories = bertNLClassifier.classify(text: "This is a test!")
        print(categories)
    }

    var body: some Scene {
        WindowGroup {
            ContentView()
        }
    }
}
```

output:
```
Warning: Error creating LLDB target at path '/Users/pkgoogle/Library/Developer/Xcode/DerivedData/test60743-axhdejqtimuqkebnrdttlwvakvhx/Build/Products/Debug-iphonesimulator/test60743.app'- using an empty LLDB target which can cause slow memory reads from remote devices.
2023-06-20 15:51:30.896779-0700 test60743[88128:5295809] Initialized TensorFlow Lite runtime.
INFO: Initialized TensorFlow Lite runtime.
2023-06-20 15:51:30.903566-0700 test60743[88128:5295809] Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
["51": 2.849037394891951e-10, "22": 1.445723984971892e-09, "26": 4.312332283797071e-13, "37": 0.9999986886978149, "20": 3.465258942014771e-07, "152": 5.632567762897667e-12, "23": 6.763575024484533e-14, "36": 4.274391396386079e-10, "29": 3.381838142857418e-10, "50": 8.567663939418324e-12, "151": 6.112114980805217e-11, "101": 2.107856389432783e-10, "228": 9.216152534463617e-07, "12": 2.11458517362928e-10, "28": 5.816923678847452e-11, "2": 3.674009707577142e-11, "39": 7.267813906253195e-09, "30": 4.441291223677979e-10, "116": 1.436604741417158e-11, "9": 1.725275478037247e-10, "1": 1.809406964536908e-11, "192": 3.250403002308389e-10, "24": 2.052141009206698e-12, "3": 8.08961964082755e-09, "137": 1.394123857155483e-12]
```

Can you check if you installed the podfiles properly? Do you all run into the same problem with a minimal example? (Feel free to use this toy example app to test). Hi @pkgoogle,

I built the same example and had the same error..
This it the output:
TestTextClassifierSwiftUI[33808:264743] Initialized TensorFlow Lite runtime.
tensor->bytes == bytes
FATAL

I attached the sample I did (without the pods folder)
[TestTextClassifierSwiftUI.zip](https://github.com/tensorflow/tensorflow/files/11817727/TestTextClassifierSwiftUI.zip)

Not sure if related... which Xcode version are you using?
 Hi @pauljasser,

I'm using Xcode 14.3.1, good news, I am able to run into your issue w/ your project. The only thing I can spot that is different about yours and my toy app is the extra frameworks in the "Pods" folder. I tried removing the frameworks but that doesn't seem to fix it. Can you start a fresh project and follow these directions for including 'TensorFlowLiteTaskText'.

1. Go to your root project directory and do:
```
pod init
```

2. Open your Podfile and make it like this:
```ruby
# Uncomment the next line to define a global platform for your project
# platform :ios, '9.0'

target 'YourProjectName' do
  # Comment the next line if you don't want to use dynamic frameworks
  use_frameworks!

  # Pods for YourProjectName
  pod 'TensorFlowLiteTaskText'
end
```

3. install the pod:
```
pod install
```

Add the model.tflite asset to your project and try running the toy app. This is to test to make sure that this works without the extra frameworks.

If that works, try adding the frameworks one by one so we can pinpoint what's causing the problem. hey @pkgoogle.. I finally found the issue.
The TensorFlowLiteTaskText library v 0.2.0 installed as described here [https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier#step_1_import_cocoapods](url) is crashing with the model we are using.
However it is working fine when using the latest version 0.4.3.
Thanks for your support. @pauljasser,

Awesome! Does this resolve the issue with android as well? If so, please feel free to close if you have no additional items. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60743">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60743">No</a>

Hello, I hope all is well.
I have recently created a MobileBERT model using the Python API of the tflite_model_maker library following the steps described in this [page](https://www.tensorflow.org/lite/models/modify/model_maker/text_classification). You may find it attached in the following [link](https://drive.google.com/file/d/1jeKm7EesBZqi_lgPrCSq_HwPapX54OlL/view?usp=sharing) (let me know if I can share it in any other way).

The good news is that I have been able to run inference with the Python API by following the steps described in this [article](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier#step_2_using_the_model). However, my colleague has been encountering issues when trying to run the same model using the Swift and Android APIs after following this [page](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier). 

After some time, he was effortlessly able to run inference on a different model made available through one of the sample apps found [here](https://storage.googleapis.com/download.tensorflow.org/models/tflite/task_library/text_classification/android/mobilebert.tflite).
It is our belief that the issue may be from the MetaData but after inspecting both models' metadata (attached in the zip file), they seem to be exactly the same: [MetaDatas.zip](https://github.com/tensorflow/tensorflow/files/11615911/MetaDatas.zip).

Our issue is that the inference process is killed without clear reason after the model is loaded. As displayed in the image below:

![MicrosoftTeams-image](https://github.com/tensorflow/tensorflow/assets/59343296/cd826cae-0544-4a2c-8f7b-8fe967466bf1)
With the following error:
```
tensor->bytes == bytes
FATAL
```

Please advise and thank you for your time