Tensorflow Lite library is crashing in WASM library at 3rd inference
@L1onKing Thanks for reporting the issue.

Have you checked with latest stable version TF 2.12 and see if the issue still exists?

Thanks. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. Hello @pjpratik! Thank you for your response.

I am trying to figure out how to compile it using bazel. Bazel is a new thing to me, so far I don't understand how to work with it. Can you maybe guide me to some tutorial that explains how to build TFLite for WASM using Bazel? Thank you very much! Hi @L1onKing 

For C++ library, the TFLite provides `libtensorflowlite.so` which can be built using `bazel build --config=elinux_aarch64 -c opt //tensorflow/lite:libtensorflowlite.so`. I've created a sample gist to build the same using bazel which can be found [here](https://colab.research.google.com/gist/pjpratik/1d1e24c386f42b4f483732dd324d418c/60210.ipynb). Please refer to cross-compilation for ARM with Bazel [here](https://www.tensorflow.org/lite/guide/build_arm#cross-compilation_for_arm_with_bazel).

Also, the TFLite supports for Tensorflow.js in which under the hood, the TFLite C++ runtime is packaged in a set of WASM modules with bazel. Please refer to [this](https://github.com/tensorflow/tfjs/tree/master/tfjs-tflite#tflite-support-for-tensorflowjs) documentation for the detailed use and let us know if it helps.

And, please find the relevant thread #46359 

Thanks. bazel build --config=elinux_aarch64 -c opt //tensorflow/lite:libtensorflowlite.so - so to my understanding, I need to run a command of such format from tensorflow repo directory. I have three questions on which I struggle to find an answer:

1. What is the `config` value if I want to build libtensorflowlite.a with WASM?
2. How to enable XNNPack during this build?
3. Where to find relevant public headers that I need to include in my project?

@pjpratik Thank you very much for your help! Hi @L1onKing 

>What is the config value if I want to build libtensorflowlite.a with WASM?

The `config` value for WASM is not available in `.bazelrc` afaik.  We need to build the Bazel Emscripten toolchain manually using the instructions given [here](https://github.com/emscripten-core/emsdk/tree/master/bazel#bazel-emscripten-toolchain) and set `--confg=wasm`.  Also, the bazel build generates the `.so` sharable dynamic object file instead of `.a` as CMAKE does.

>How to enable XNNPack during this build?

The XNNPACK can be enable by adding `--define tflite_with_xnnpack=true` during bazel build.

As this [comment](https://github.com/tensorflow/tensorflow/issues/46359#issuecomment-1215595828) suggests,  try the TFLite with TFjs which uses WASM backend given the bazel support for which comes with pre built binaries.


Thanks.

 Hi @pjpratik ! 

Thank you for your recommendations, for the time being I am exploring this - https://github.com/tensorflow/tfjs/tree/master/tfjs-tflite

So what I did:
1. Downloaded this repo
2. Navigated to tfjs-tflite folder where BUILD.bazel is located
3. Ran the command `bazel build //tfjs-tflite:tf-tflite`

But I am having next output

```
INFO: Repository npm instantiated at:
  /home/nabeel/Documents/algoface/tfjs/WORKSPACE:67:13: in <toplevel>
  /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/index.bzl:83:18: in yarn_install
Repository rule yarn_install defined at:
  /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:1002:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'npm':
   Traceback (most recent call last):
	File "/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 866, column 40, in _yarn_install_impl
		yarn_version = _detect_yarn_version(repository_ctx, yarn_cmd)
	File "/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 844, column 13, in _detect_yarn_version
		fail("yarn --version failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: yarn --version failed:  (/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node)
)
ERROR: /home/nabeel/Documents/algoface/tfjs/WORKSPACE:67:13: fetching yarn_install rule //external:npm: Traceback (most recent call last):
	File "/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 866, column 40, in _yarn_install_impl
		yarn_version = _detect_yarn_version(repository_ctx, yarn_cmd)
	File "/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 844, column 13, in _detect_yarn_version
		fail("yarn --version failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: yarn --version failed:  (/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node)
)
ERROR: Skipping '//tfjs-tflite:tf-tflite': no such package '@npm//@bazel/concatjs': yarn --version failed:  (/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node)
)
WARNING: Target pattern parsing failed.
ERROR: no such package '@npm//@bazel/concatjs': yarn --version failed:  (/home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/nabeel/.cache/bazel/_bazel_nabeel/01ad2d90cd2af4a5ee7bc50cb9e6914b/external/nodejs_linux_amd64/bin/nodejs/bin/node)
)
INFO: Elapsed time: 3.087s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tfjs-tflite
```
It looks like the issue is in incompatibility of bazel and glibc.

The output of `bazel version` on my machine:

```
Build label: 5.3.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Aug 23 00:45:53 2022 (1661215553)
Build timestamp: 1661215553
Build timestamp as int: 1661215553

```

Could you please advise what version of bazel should I use in order to make it run? Thank you very much!
 Hi @L1onKing 

The Bazel version 5.3.0 should be compatible. Moreover, you can build the `tfjs-tflite` using `yarn build` which inturn uses bazel with wasm backend.

- Navigate to `tfjs-tflite` repo
- Check for `yarn`  installation
- Run `yarn build`

Please check this [gist](https://colab.research.google.com/gist/pjpratik/35f081254f9a619cc386b1cd085f059a/60210.ipynb) provided with the `tfjs-tflite` build and can be tested  with`yarn test` with successful build.

Thanks.
 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. Hi @pjpratik ! Sorry for a long response, I was unavailable last week. Actually I did try `yarn build` command from `tfjs-tflite` folder, but that is the output I got:

tfjs/tfjs-tflite$ yarn build
00h00m00s 0/0: : ERROR: [Errno 2] No such file or directory: 'build'

yarn is installed and available in Terminal. Could you advise please? **UPDATE**

I have figured an issue with yarn and now `yarn build` command executes, but I have exactly the same issue:

`libc.so.6: version `GLIBC_2.28' not found`


Please advise how to fix it Hi @L1onKing 

Can you please check if latest `npm` and `nodejs` versions are installed?

Thanks. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60210">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60210">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Support

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.7.0

### Custom Code

Yes

### OS Platform and Distribution

Emscripten, Ubuntu 18.04

### Mobile device

_No response_

### Python version

_No response_

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
Hello! I have C++ code that I want to deploy as WASM library and this code contains TFLite library. I have compiled TFLite library with XNNPack support using Emscripten toolchain quite easy, so no issue there. I have a leight-weight convolution+dense model that runs perfectly on Desktop, but I am starting having problems in the browser.

In 99% of cases I have an error on the third inference:

Uncaught RuntimeError: memory access out of bounds

Through some trivial debugging I have found out that the issue comes from _interpreter->Invoke() method. Does not matter if I put any input or not, I just need to call Invoke() three times and I have a crash.

First thing first: I decided to add more memory to my WASM library by adding this line to CMake:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -s TOTAL_STACK=134217728 -s TOTAL_MEMORY=268435456")
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -s TOTAL_STACK=134217728 -s TOTAL_MEMORY=268435456")

128 MB and 256 MB in total for 1 MB model - I think this is more than enough. And on top of that, I am allowing Memory Growth. But unfortunately, I have exactly the same issue.

I am beating on this problem for 2 weeks straight and at this stage I have no clue how to fix it. Also I have tried to set custom allocation using TfLiteCustomAllocation but in this case I have a crash on the very first inference. I guess I was not using it right, but unfortunately I couldn't find even one tutorial describing how to apply custom allocation in TFLite.

I said that I have a crash in 99% of cases. There was one time when WASM library worked and inference worked as well. It happens just randomly once, and I couldn't reproduce it anymore.
```


### Standalone code to reproduce the issue

```shell
Here is the code that does TFLite inference


#include <cstdlib>
#include "tflite_model.h"
#include <iostream>

#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/util.h"

namespace tracker {

#ifdef EMSCRIPTEN
	void TFLiteModel::init(std::stringstream& stream) {

		std::string img_str = stream.str();
		std::vector<char> img_model_data(img_str.size());
		std::copy(img_str.begin(), img_str.end(), img_model_data.begin());

		_model = tflite::FlatBufferModel::BuildFromBuffer(img_str.data(), img_str.size());
#else
	void TFLiteModel::init(const std::string& path) {
		_model = tflite::FlatBufferModel::BuildFromFile(path.c_str());

#endif

		tflite::ops::builtin::BuiltinOpResolver resolver;
		tflite::InterpreterBuilder(*_model, resolver)(&_interpreter);

		_interpreter->AllocateTensors();

		/*for (int i = 0; i < _interpreter->tensors_size(); i++) {
			TfLiteTensor* tensor = _interpreter->tensor(i);

			if (tensor->allocation_type == kTfLiteArenaRw || tensor->allocation_type == kTfLiteArenaRwPersistent) {

				int aligned_bytes = tensor->bytes + (tflite::kDefaultTensorAlignment - tensor->bytes % tflite::kDefaultTensorAlignment) % tflite::kDefaultTensorAlignment;

				TfLiteCustomAllocation customAlloc;
				int result = posix_memalign(&customAlloc.data, tflite::kDefaultTensorAlignment, tensor->bytes);
				if (result != 0 || customAlloc.data == NULL) {
					std::cout << "posix_memalign does not work!\n";
				}

				TfLiteStatus st = _interpreter->SetCustomAllocationForTensor(i, customAlloc);
				std::cout << "status = " << st << std::endl;
				if (tensor->bytes % tflite::kDefaultTensorAlignment != 0) {
					std::cout << "bad! i " << i << ", size " << tensor->bytes << std::endl;
				}
				_allocations.push_back(customAlloc);
			}
		}
		exit(0);*/
	}

	void TFLiteModel::forward(const cv::Mat& img_input, const std::vector<float>& lms_input) {

		float* model_in = _interpreter->typed_input_tensor<float>(0);
		std::memcpy(model_in, img_input.data, img_input.total() * img_input.elemSize());

		float* lms_in = _interpreter->typed_input_tensor<float>(1);
		std::memcpy(lms_in, lms_input.data(), sizeof(float) * lms_input.size());
		
		_interpreter->Invoke();
	}

	float* TFLiteModel::out() {
		return _interpreter->typed_output_tensor<float>(0);
	}

	std::vector<int> TFLiteModel::getOutputShape() const {
		TfLiteTensor* outtensor = _interpreter->output_tensor(0);
		TfLiteIntArray* dims = outtensor->dims;

		std::vector<int> sh;
		for (int i = 0; i < dims->size; i++) {
			sh.push_back(dims->data[i]);
		}

		return sh;
	}
}
```


### Relevant log output

_No response_</details>