OperatorNotAllowedInGraphError: Iterating over a symbolic `tf.Tensor` is not allowed when using a dataset with tuples
The bug in the code is that you are using the `fit` method of the `tf.keras.Model` with the `train_data` dataset, but `train_data` is not defined anywhere in the code.

Solution:
```
model.fit(dataset, batch_size=1, epochs=1)
```

Additionally, the `call` method of your custom `MyTransformer` model is not returning the labels `lable` which is required for the training process. You'll need to return both the features and the labels from the `call` method in order for the model to be trained properly:
```
def call(self, inputs, training):
    feature, lable = inputs
    return feature, lable
```
Finally, the loss function that you are using, `tf.keras.losses.BinaryCrossentropy` is designed for binary classification problems, but the labels in your dataset are categorical. To handle this, you should use a different loss function, such as `tf.keras.losses.CategoricalCrossentropy`, which is designed for categorical classification problems. Let me know if it helps ? Hi @DURGESH716, and thank you for the reply.

using the wrong variable was a mistake that I made when pasting the example here. Even if a not defined variable was used I would've expected "NameError: name 'train_data' is not defined" error and not one for a symbolic tensor. The link that was provided with the description of the issue points out to google colab example where the correct variable was used. So the original question still stands. 
I've now modified the example according to your suggestions to return feature and label, plus using the CategoricalCrossentropy and nothing changes. Here is a link to the example in google colab:
**https://colab.research.google.com/drive/1mn6iseJLnJwTmwakYa2XuxszKtR6sV9G#scrollTo=Cj9g0bGN1Fo3**

Can you help me to understand what the real issue is? @mihail-vladov Thanks for reporting issue.

@sushreebarsa  I was able to reproduce this issue. Please find the gist [here](https://colab.research.google.com/gist/pjpratik/11bc8b59d3a76c5a3619e6091e35f295/59510.ipynb). 

The model works as expected when features and labels are given as forward pass but fail when `tf.data.Dataset` is passed to the `model.fit`. Proceeding with the @pjpratik points, I think it is important to ensure that the data is correctly formatted and preprocessed before being passed to the model. Some common issues include incorrect tensor shapes, data types, and normalization. To resolve the issue, you may need to inspect and debug the data preprocessing steps, including any data augmentation or batching operations that are applied to the data before passing it to the model. Additionally, it may be helpful to compare the data and preprocessing steps used when passing features and labels directly to the model versus when using a tf.data.Dataset.

Rest everything logic and code is syntactically correct, please check the dataset properly and pre-process it carefully.  Hi, @mihail-vladov 

Apologize for the delay and It seems like you haven't done pre-processing with your dataset, your `batched_labels` should be in `numerical values` but it was in `string` type so you'll have to do some pre-processing with `batched_labels` with one hot encoding approach so that `batched_labels`  will convert into numerical values so you can refer this [official documentation ](https://www.tensorflow.org/api_docs/python/tf/one_hot)

When you pass dataset from `tf.data.Dataset()` to model.fit() it is expected that it will directly generate batches, not individual examples. You just need to batch your dataset before training as shown below and please refer [tf.data.Dataset()](https://www.tensorflow.org/api_docs/python/tf/data/Dataset):

```
batch_size=32
dataset = dataset.batch(batch_size)
model.fit(x=dataset)
```
I have added sample code example for your reference, How to use `tf.data.dataset` with `model.fit()` so please refer this [gist-file](https://colab.research.google.com/gist/gaikwadrahul8/d29fb5cebf247c702d6d36ae34bd2859/-59510.ipynb) and also refer our official documentation for [Customize what happens in Model.fit](https://www.tensorflow.org/guide/keras/customizing_what_happens_in_fit) 

I hope it will help you to resolve your issue and if issue still persists please let us know ? or Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ? Thank you! Hi @gaikwadrahul8,

I'm a bit confused by your response, as the code in your gist-file appears to differ significantly from the code I provided, except for the use of TensorFlow and a dataset. Additionally, @pjpratik has confirmed that there is a bug, so I was hoping to receive an update on when the bug will be fixed.

Regarding your previous comment about the data preprocessing and batches, I understand your point and you would be typically right if there was no issue and I was actually able to use the data. However, this is not the case. Nevertheless, I've reworked the example to reflect your comments. Here is the updated [example](https://colab.research.google.com/drive/1OO4PvlDNSyNKKVyvXwOLneDdMm2oY8ho). As you can verify yourself, the exception is still there.

I would like to reiterate the issue I am encountering. According to the [TensorFlow documentation](https://www.tensorflow.org/api_docs/python/tf/keras/Model#call), I should be able to use a dataset as the **x** argument for the model.fit() function and receive a tuple as **inputs** parameter in the model.call() function. Then, I would decide how to process and what operation to execute on the data in that function. However, I'm encountering an exception and can't obtain the data in the tuple format from the inputs parameter in the call function. This leaves me with the impression that the documentation is not accurate! Meaning there is a bug. Here is once again the exception: 

`File "<ipython-input-7-a0afc8cfb653>", line 8, in call  
(feature, label) = inputs`
`OperatorNotAllowedInGraphError: Iterating over a symbolic tf.Tensor is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.`

If you decide to provide an example of how to overcome the exception please use the code I've provided with the specified model and a dataset containing a tuple. Thank you for your time and consideration.
 This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.
 Hi @gaikwadrahul8,
is there any update on this? Hi, @mihail-vladov 

Apologize for the delay and It seems like you're trying to use operation which does not support by `autograph` because of that reason you're getting that error and even error itself is saying the same thing but there is some workaround and I found similar issue over [stack-overflow](https://stackoverflow.com/questions/68636916/operatornotallowedingrapherror-iterating-over-tf-tensor-is-not-allowed-autog). and it seems like you'll have to use `tf.shape(x)`, please refer this [official documentation](https://www.tensorflow.org/api_docs/python/tf/shape), please try with `tf.shape(x)` and check whether your issue is resolving or not ? I tried from my end and I'm getting different error `AttributeError: 'NoneType' object has no attribute 'dtype'` and I have added [gist-file](https://colab.research.google.com/gist/gaikwadrahul8/4f3a16cd77901db790666fffd16d64ff/-59510-test.ipynb) for your reference. I hope it will help you to resolve your issue.

`tf.shape(x) and x.shape should be identical in eager mode. Within tf.function, not all dimensions may be known until execution time. Hence when defining custom layers and models for graph mode, prefer the dynamic tf.shape(x) over the static x.shape.`

If issue still persists please let us know ? Thank you! Hi, @gaikwadrahul8,

pjpratik already confirmed that this is a **bug**. Why are you trying to convince me that I am doing something incorrectly?

You said that I am trying to use an operation that is not supported. **Could you please point out exactly which is that operation?**

Regarding your comment on the different error you are getting, you have changed the code in the model, which is why the error is different.

I will repeat myself again. According to the TensorFlow documentation, I can use the dataset as input data. This data can be packed as a tuple. Please extract the data from the tuple in the model call function in a way that won't throw an error. Hi, @mihail-vladov 

Apologize for the delayed response and I see @pjpratik did not say explicitly it's bug, He only replicated that issue from his end and he said code execution is failing with `tf.data.Dataset` is passed to the `model.fit()` and I was referring there are similar issues with same error message while using `Autograph` mode like [#32546](https://github.com/tensorflow/tensorflow/issues/32546), [#51472](https://github.com/tensorflow/tensorflow/issues/51472) and users found some workaround for those issues.

I'm waiting for response from concerned team for this issue whether is there any workaround to handle this issue till then could you please have look into this [Autograph reference guide](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/index.md) which may help you to solve this issue and I'll update you soon here. Thank you! Hi, @SuryanarayanaY 

Could you please look into this issue ? Thank you! Hi @mihail-vladov ,

The problem not seems to related to Autograph. The problem seems the code trying to get Two values(feature, lable) from a Tensor (`model.fit` converts the given `input` to `Tensor` that is internal details can be checked from the `model.fit` API code) and Tensor can't be unpacked to Two values and Please note that Tensor is immutable in Tensorflow.

You can check the behaviour clearly when you enable Eager mode using `model.compile(...,run_eagerly=True)` where you can see the error below.

`not enough values to unpack (expected 2, got 1)`

I have added `print(inputs)` in `call()` to understand what actually is `inputs`. Please refer attached [gist](https://colab.research.google.com/gist/SuryanarayanaY/86c052333f39f95c63e3e194c7292c60/59510.ipynb).

Hope this will clear your doubts on root cause of this error. Thanks! Hi @SuryanarayanaY, Thank you for the answer.

According to the [TensorFlow documentation ](https://www.tensorflow.org/api_docs/python/tf/keras/Model#call)using tuple (feature, label) as inputs argument is allowed. Here is the quote: 

> inputs:	Input tensor, or dict/list/tuple of input tensors.

I am using a tuple. Here are my questions:

1. Why the model.fit function does not call the model.call function according to the documentation?
2. What is the correct way to obtain the feature and label data in the model.call function when I pass a dataset to the model.fit function?



 Hi @mihail-vladov ,

> 1. Why the model.fit function does not call the model.call function according to the documentation?

The model.fit function called the call() function. You can check by adding print() to confirm same. But here when we pass a dataset as an argument to model.fit, the API converts it into Tensors internally.Outside the model.fit() the dataset might be a tuple but within model.fit the tuple is converting into Tensors which is default behaviour.




> 2\. What is the correct way to obtain the feature and label data in the model.call function when I pass a dataset to the model.fit function?

To get the custom behaviour as per individuals requirement you need to override the `train_step` . Please refer to attached [tutorials](https://www.tensorflow.org/guide/keras/customizing_what_happens_in_fit#a_first_simple_example) for more details.

Thank you! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59510">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59510">No</a>
 I am facing the same issue and I am confused why the example in [Neural Machine Translation](https://www.tensorflow.org/text/tutorials/nmt_with_attention) doesn't run into the same issue but here we run into this issue? Basically they are using the same type of code:
```
class Translator(tf.keras.Model):
  @classmethod
  def add_method(cls, fun):
    setattr(cls, fun.__name__, fun)
    return fun

  def __init__(self, units,
               context_text_processor,
               target_text_processor):
    super().__init__()
    # Build the encoder and decoder
    encoder = Encoder(context_text_processor, units)
    decoder = Decoder(target_text_processor, units)

    self.encoder = encoder
    self.decoder = decoder

  def call(self, inputs):
    context, x = inputs
    context = self.encoder(context)
    logits = self.decoder(context, x)

    #TODO(b/250038731): remove this
    try:
      # Delete the keras mask, so keras doesn't scale the loss+accuracy. 
      del logits._keras_mask
    except AttributeError:
      pass

    return logits
``` @kiskani This was one of the issues that made me switch to PyTorch. Misleading documentation and bad support from the developers here. I do not regret my decision.
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

No

### Source

source

### Tensorflow Version

2.11

### Custom Code

Yes

### OS Platform and Distribution

Windows

### Mobile device

_No response_

### Python version

3.9

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
I am trying to create my own transformer and train it. For this purpose, I use dataset to handle my data. The data is created by a code snippet from the tensorflow dataset.from_tensor_slices() method [documentation article](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_tensor_slices) . Nevertheless, tensorflow is giving me the following error when I call the fit() method:

> "OperatorNotAllowedInGraphError: Iterating over a symbolic tf.Tensor is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature."

The used code is reduced significantly just for the purpose of reproducing the issue.

I've also tried passing the data as a dictionary instead of a tuple in the dataset and a couple more things but nothing worked. It seems that I am missing something.
Here is a link to [google colab example](https://colab.research.google.com/drive/1mn6iseJLnJwTmwakYa2XuxszKtR6sV9G#scrollTo=Cj9g0bGN1Fo3)
```


### Standalone code to reproduce the issue

```shell
import numpy as np
import tensorflow as tf

batched_features = tf.constant([[[1, 3], [2, 3]],
                                [[2, 1], [1, 2]],
                                [[3, 3], [3, 2]]], shape=(3, 2, 2))
batched_labels = tf.constant([['A', 'A'],
                              ['B', 'B'],
                              ['A', 'B']], shape=(3, 2, 1))
dataset = tf.data.Dataset.from_tensor_slices((batched_features, batched_labels))
dataset = dataset.batch(1)
for element in dataset.as_numpy_iterator():
  print(element)

class MyTransformer(tf.keras.Model):
    def __init__(self):
        super().__init__()
        
    def call(self, inputs, training):
        print(type(inputs))
        feature, lable = inputs
        return feature

model = MyTransformer()
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
              loss=tf.keras.losses.BinaryCrossentropy(),
              metrics=[tf.keras.metrics.BinaryAccuracy(),
                       tf.keras.metrics.FalseNegatives()])

model.fit(dataset , batch_size = 1, epochs = 1)
```


### Relevant log output

_No response_</details>