JIT compilation failed.
This is a fairly old version of TF by now, did you try with a newer one? > This is a fairly old version of TF by now, did you try with a newer one?

I am on windows native and TF 2.10 is the latest version on TensorFlow that support CUDA on windows. So, I am with it. Hi **@Aditya-Singh-3112** ,

Could you make sure the frames, mean, and std variables are of compatible types before the subtraction and division. You are already casting frames to tf.float32 during normalization, but ensure that mean and std are also cast to tf.float32 to avoid any type mismatch issues.

```
mean = tf.math.reduce_mean(frames, axis=[0, 1, 2], keepdims=True)
std = tf.math.reduce_std(tf.cast(frames, tf.float32), axis=[0, 1, 2], keepdims=True)
frames = tf.cast(frames, tf.float32)
normalized_frames = (frames - mean) / std
```

Make sure to adjust the axis parameter in reduce_mean and reduce_std if necessary, depending on the shape of your frames tensor.

Thank you! Hello @Venkat6871 ,

I casted the mean, frames and std to tf.float32 before normalizing, as you advised and it worked perfectly.

Thank you very much.  I will close this issue now. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/63167">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/63167">No</a>

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

source

### TensorFlow version

2.10.10

### Custom code

Yes

### OS platform and distribution

Windows 11

### Mobile device

No

### Python version

3.10

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

11.8

### GPU model and memory

NVIDIA RTX 3060 12 GB

### Current behavior?

The code should have loaded the val data.

### Standalone code to reproduce the issue

```shell
import os
import cv2
import tensorflow as tf
import numpy as np
import typing as List
import matplotlib.pyplot as plt
import imageio

physical_devices = tf.config.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)

import gdown
url = 'https://drive.google.com/uc?id=1YlvpDLix3S-U8fd-gqRwPcWXAXm8JwjL'
output = 'data.zip'
gdown.download(url, output, quiet=False)
gdown.extractall('data.zip')

vocab = [x for x in "abcdefghijklmnopqrstuvwxyz'?!123456789 "]
char_to_num = tf.keras.layers.StringLookup(vocabulary=vocab, oov_token="")
num_to_char = tf.keras.layers.StringLookup(
    vocabulary=char_to_num.get_vocabulary(), oov_token="", invert=True
)

def load_video(path:str): 

    cap = cv2.VideoCapture(path)
    frames = []
    for _ in range(int(cap.get(cv2.CAP_PROP_FRAME_COUNT))): 
        ret, frame = cap.read()
        frame = tf.image.rgb_to_grayscale(frame)
        frames.append(frame[190:236,80:220,:])
    cap.release()
    mean = tf.math.reduce_mean(frames)
    std = tf.math.reduce_std(tf.cast(frames, tf.float32))
    return tf.cast((frames - mean), tf.float32) / std
  
def load_alignments(path:str): 
    with open(path, 'r') as f: 
        lines = f.readlines() 
    tokens = []
    for line in lines:
        line = line.split()
        if line[2] != 'sil': 
            tokens = [*tokens,' ',line[2]]
    return char_to_num(tf.reshape(tf.strings.unicode_split(tokens, input_encoding='UTF-8'), (-1)))[1:]

def load_data(path: str): 
    path = bytes.decode(path.numpy())
    file_name = path.split('\\')[-1].split('.')[0]
    video_path = os.path.join('data','s1',f'{file_name}.mpg')
    alignment_path = os.path.join('data','alignments','s1',f'{file_name}.align')
    frames = load_video(video_path) 
    alignments = load_alignments(alignment_path)
    
    return frames, alignments

def mappable_function(path:str):
    result = tf.py_function(load_data, [path], (tf.float32, tf.int64))
    return result

data = tf.data.Dataset.list_files('./data/s1/*.mpg')
data = data.shuffle(500, reshuffle_each_iteration=False)
data = data.map(mappable_function)
data = data.padded_batch(2, padded_shapes=([75,None,None,None],[40]))
data = data.prefetch(tf.data.AUTOTUNE)
# Added for split 
train = data.take(450)
test = data.skip(450)

sample = data.as_numpy_iterator()

val = sample.next(); val[0]
```


### Relevant log output

```shell
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
Cell In[16], line 1
----> 1 val = sample.next(); val[0]

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:4638, in _NumpyIterator.next(self)
   4637 def next(self):
-> 4638   return self.__next__()

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:4635, in _NumpyIterator.__next__(self)
   4632     numpy.setflags(write=False)
   4633   return numpy
-> 4635 return nest.map_structure(to_numpy, next(self._iterator))

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py:766, in OwnedIterator.__next__(self)
    764 def __next__(self):
    765   try:
--> 766     return self._next_internal()
    767   except errors.OutOfRangeError:
    768     raise StopIteration

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py:749, in OwnedIterator._next_internal(self)
    746 # TODO(b/77291417): This runs in sync mode as iterators use an error status
    747 # to communicate that there is no more data to iterate over.
    748 with context.execution_mode(context.SYNC):
--> 749   ret = gen_dataset_ops.iterator_get_next(
    750       self._iterator_resource,
    751       output_types=self._flat_output_types,
    752       output_shapes=self._flat_output_shapes)
    754   try:
    755     # Fast path for the case `self._structure` is not a nested structure.
    756     return self._element_spec._from_compatible_tensor_list(ret)  # pylint: disable=protected-access

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py:3016, in iterator_get_next(iterator, output_types, output_shapes, name)
   3014   return _result
   3015 except _core._NotOkStatusException as e:
-> 3016   _ops.raise_from_not_ok_status(e, name)
   3017 except _core._FallbackException:
   3018   pass

File ~\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\framework\ops.py:7209, in raise_from_not_ok_status(e, name)
   7207 def raise_from_not_ok_status(e, name):
   7208   e.message += (" name: " + name if name is not None else "")
-> 7209   raise core._status_to_exception(e) from None

UnknownError: {{function_node __wrapped__IteratorGetNext_output_types_2_device_/job:localhost/replica:0/task:0/device:CPU:0}} UnknownError: {{function_node __wrapped__Sub_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Sub]
Traceback (most recent call last):

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 269, in __call__
    return func(device, token, args)

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 147, in __call__
    outputs = self._call(device, args)

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 154, in _call
    ret = self._func(*args)

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 642, in wrapper
    return func(*args, **kwargs)

  File "C:\Users\adity\AppData\Local\Temp\ipykernel_9688\3954971381.py", line 6, in load_data
    frames = load_video(video_path)

  File "C:\Users\adity\AppData\Local\Temp\ipykernel_9688\1209296760.py", line 13, in load_video
    return tf.cast((frames - mean), tf.float32) / std

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None

  File "C:\Users\adity\miniconda3\envs\tfgpu\lib\site-packages\tensorflow\python\framework\ops.py", line 7209, in raise_from_not_ok_status
    raise core._status_to_exception(e) from None  # pylint: disable=protected-access

tensorflow.python.framework.errors_impl.UnknownError: {{function_node __wrapped__Sub_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Sub]


	 [[{{node EagerPyFunc}}]] [Op:IteratorGetNext]
```
