How/Where decision is made to choose between eager execution & graph execution for Tensorflow kernel OPs
Hi @penpornk, @tilakrayal , @sachinprasadhs, @TensorFlow-MKL, @huiyan2021,
Can i get any suggestions/help/answer regarding the question? @vineel96 
In TF1.x, model.fit() always run in graph model, while in TF2.x eager mode is enabled by default. You can disable eager mode by **tf.compat.v1.disable_eager_execution()**. How did you train the model? Could you share the code and the whole log? Thanks!

 Hi @huiyan2021,

**My Doubt:** If we concentrate only on the execution of OP kernels (MKL and non MKL) in the log file **after fit() function** of model, For **non MKL kernel OP's** (AddV2,Mul) there is  eager execution & graph execution  and for **MKL OP Kernel's** (_MKLMatMul) I observe only graph execution & no eager execution.  Question is how decision is made to which kernel OP's are executed in eager mode & graph mode.
**Other observation:** For transpose operation, I observe two kernels for it: normal Transpose OP and _MKLTranspose OP, where normal Transpose OP executes in eager mode and _MKLTranspose OP executes in graph mode. So either of the one OP should be present, why there is two kernel OP's are present? 

**Train Code:**
Set: export TF_CPP_MAX_VLOG_LEVEL=2
       export ONEDNN_VERBOSE=1
```
from transformers import TFBertForSequenceClassification
from transformers import BertTokenizer, glue_convert_examples_to_features
import tensorflow as tf
import tensorflow_datasets as tfds
import datetime

logdir = "logs/fit/"
tf.debugging.experimental.enable_dump_debug_info(logdir, tensor_debug_mode="FULL_HEALTH", circular_buffer_size=-1)

tf.config.threading.set_inter_op_parallelism_threads(1)
tf.config.threading.set_intra_op_parallelism_threads(1)

model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
data, info = tfds.load('glue/mrpc',with_info=True)
train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
#print(info)
train_dataset = train_dataset.shuffle(100).batch(1).repeat(1) # batch 32
 

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=optimizer, loss=loss)

log_dir = logdir + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1,profile_batch=1)


model.fit(train_dataset, epochs=1, steps_per_epoch=1,callbacks=[tensorboard_callback]) 
```
I attach two log files:
**_bert_pretrain.txt:_**  This logs are only from bert model object creation and weight loading line only.
**_bert_recent.txt:_** This logs are from model creation and model fitting , full model fitting for 1 epoch & 1 steps per epoch

**_bert_pretrain.txt:_** https://drive.google.com/file/d/1TDHrKr6ccCOEmQNVPny6B1Vm7P2NrIBz/view?usp=drive_link
**_bert_recent.txt:_** :https://drive.google.com/file/d/1WJyP7r8B4C8wZmyumw1kJhiPDHncvmAb/view?usp=drive_link 

**Observation:**
1.  From **_bert_pretrain.txt_**, I see both MKL and non MKL kernel OP's executing **only** in **eager mode.**
2.  From  **_bert_recent.txt_** , If we filter out logs corresponding to starting from model.fit() function, I observe only graph execution for MKL kernel OP's and both eager and graph execution for non MKL kernel OP's.
Why two modes of execution (eager and graph) for non MKL kernel OP's ? 

 Hi @sachinprasadhs, @vineel96

It is better that someone from Google answers this question. Model could have a mixed execution between eager and graph. TF decides which ops are part of the graph so it runs in graph mode, and which runs in eager mode. Intel code does not control that decision, we replace an op by _Mkl* only after it was decided by TF to run eagerly or in graph mode. Also, not all ops are supported by Mkl (oneDNN) in both eager and graph mode. For example, Transpose op is only supported by Mkl in graph mode not in eager mode. So if it runs in eager mode, it will be non-mkl Transpose op. Hi @huiyan2021,
Thanks for the answer.
Can we know the reason for why a MKL op is choosen to run in graph mode only or eager mode only or in both eager and graph mode? If MKL op is supported in both eager and graph modes, which mode is selected to run the MKL op finally? Is there any file location where this decision is made?

**My understanding:**  
_**(Pls refer below figure)**_
There are two phases: 1.) graph optimization and 2.) graph execution. In graph optimization, there are two modes: eager execution and MKL layout rewrite propagation. TF OP to MKL OP conversion takes place if MKL support is there for that, if there is no equivalent MKL support for the op  then it optimizes in eager mode.  In graph execution, the actual execution of MKL and non MKL ops takes place. 
![image](https://github.com/tensorflow/tensorflow/assets/131174187/7fbc85d4-ab7f-4ab4-989d-4a79657d08cd)

**Is my understanding correct?**

Also tagging google members @sachinprasadhs @penpornk @TensorFlow-MKL again, to get answers through TF side. Hello @sachinprasadhs , @penpornk, @TensorFlow-MKL 
Can I get any suggestions/help/answer regarding the question?
I ran HuggingFace BERT model which uses tensorflow 2.13v with oneDNN support on intel machine and recorded its execution logs by setting TF_CPP_MAX_VLOG_LEVEL=2 & ONEDNN_VERBOSE=1 in file.

**Observation :** I have observing logs that are produced after model creation and its weight loading. Since model.fit() always run in graph model, all tensorflow kernel OPs (onednn's mkl kernel op and non-mkl kernel ops) should run in graph mode. But i observe only for non-mkl kernel ops (like ADDV2, Mul) are executing in eager mode followed by graph mode. I dont see any mkl kernel ops(like _MklMatMul) running in eager mode.

**Questions:** I want to know the reason and file where decision making is made for which op there should be eager mode. Since model.fit() runs in graph mode, why I am seeing eager mode execution for all non-mkl ops?

Sample Logs for model.fit() for ADDV2 kernel op:

```
2023-07-31 03:48:44.632289: I tensorflow/core/common_runtime/eager/execute.cc:1678] Executing op AddV2 in device /job:localhost/replica:0/task:0/device:CPU:0 --> executing addv2 eagerly After some other logs in between, I see below log:

2023-07-31 03:50:01.968512: I tensorflow/core/common_runtime/executor.cc:841] Process node: 8127 step -4458402160563696089 {{node tf_bert_for_sequence_classification/bert/encoder/layer_._0/output/LayerNorm/batchnorm/add_1}} = AddV2[T=DT_FLOAT, _XlaHasReferenceVars=false, device="/job:localhost/replica:0/task:0/device:CPU:0"](tf_bert_for_sequence_classification/bert/encoder/layer.0/output/LayerNorm/batchnorm/mul_1, tf_bert_for_sequence_classification/bert/encoder/layer._0/output/LayerNorm/batchnorm/sub) device: 
```
/job:localhost/replica:0/task:0/device:CPU:0 --> executing addv2 in graph mode i assume

**Expected to happen:** All kerenl ops should execute in graph mode.
