multiprocessing stuck after usage of tensorflow functionality
@cevheck,
I tried to execute the mentioned code on tensorflow v2.12 and it was executed without any issue/error and the output also  was expected.  Kindly find the gist of it [here](https://colab.sandbox.google.com/gist/tilakrayal/24de734f496450ad1cf18bde991a899e/untitled1186.ipynb). Thank you! @tilakrayal,
Thank you for your effort! I don't seem to get the prints from inside the multiprocessing function, however the file does completely run without errors indeed. 

However I tried the same code again when starting from a conda environment from scratch, taking the same python version as yours (3.10.11) and only downloading tensorflow. 
print(tf.__version__) returns 2.12.0, hence the same as yours.

Could this issue be hardware related? I'm working on a CPU with 16cores. One of the difference I see is you not having these prints:
"
2023-05-31 19:20:51.221681: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2023-05-31 19:20:51.221727: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cedric-Z590-UD-AC): /proc/driver/nvidia/version does not exist
2023-05-31 19:20:51.222470: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
"
 Maybe some additional information I've obtained in the meantime.

After further debugging I've seen that the issue does not arise when setting:
    tf.config.threading.set_intra_op_parallelism_threads(1)
    tf.config.threading.set_inter_op_parallelism_threads(1)
However this ofcourse is slow and hence not desired.

Additionally in debug mode I can go to the line where the file gets stuck and run this single line then in the debug console. The output I get there is the following (see below):
My hypothesis from both points is that Tensorflow is parallelizing some computations but the multiprocessing does it aswell and both don't seem to take into account eachother resulting in a deadlock situation (I assume that for the big matrix inverse additional processors are attempted to be used however those are busy in a parallel process, however I'm not very familiar with this topic). 

"""
Evaluating: Pwk = tf.transpose(Pwk) did not finish after 3.00 seconds.
This may mean a number of things:
- This evaluation is really slow and this is expected.
    In this case it's possible to silence this error by raising the timeout, setting the
    PYDEVD_WARN_EVALUATION_TIMEOUT environment variable to a bigger value.

- The evaluation may need other threads running while it's running:
    In this case, it's possible to set the PYDEVD_UNBLOCK_THREADS_TIMEOUT
    environment variable so that if after a given timeout an evaluation doesn't finish,
    other threads are unblocked or you can manually resume all threads.

    Alternatively, it's also possible to skip breaking on a particular thread by setting a
    `pydev_do_not_trace = True` attribute in the related threading.Thread instance
    (if some thread should always be running and no breakpoints are expected to be hit in it).

- The evaluation is deadlocked:
    In this case you may set the PYDEVD_THREAD_DUMP_ON_WARN_EVALUATION_TIMEOUT
    environment variable to true so that a thread dump is shown along with this message and
    optionally, set the PYDEVD_INTERRUPT_THREAD_TIMEOUT to some value so that the debugger
    tries to interrupt the evaluation (if possible) when this happens.
 """

Hopefully this extra information helps, I'm looking forward to your response.
 @cevheck,
In some of the cases there might be a chance the issue related with the Hardware and also you can safely ignore the above mentioned warnings(W) and information(I) and also it was stated that `To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags`

https://www.tensorflow.org/api_docs/python/tf/config/threading/set_intra_op_parallelism_threads

 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60741">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60741">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

No

### Source

binary

### Tensorflow Version

v2.11.0-rc2-17-gd5b57ca93e5 2.11.0

### Custom Code

Yes

### OS Platform and Distribution

Linux Ubuntu 20.04

### Mobile device

_No response_

### Python version

3.9

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

In a project where I wanted to implement multiprocessing in combination with a Tensorflow function, the processes kept getting stuck. After some debugging I was able to create a minimal example as seen below or as provided as .txt doc.
[minimal_example_MPlock.txt](https://github.com/tensorflow/tensorflow/files/11616923/minimal_example_MPlock.txt)

In words; when I do some random transpose operation in the process there is no problem at all. However afterwards if I use any functionality from tensorflow and repeat the code that used to work, all of a sudden it gets stuck on the last matrix inverse (which is quite a big one, but shouldn't be any problem).

What you can see, and is probably part of the issue, is that each time the function is called there's a bunch of tensorflow warnings. So far in all of my code I could always just ignore them, but to be sure I added them here in the logs aswell.

Thanks in advance!


### Standalone code to reproduce the issue

```shell
from multiprocessing import Process
import numpy as np
import tensorflow as tf
import time

def test_inverse():
    Pxk = tf.eye(2)
    Pwk = tf.eye(259)
    print("here1")
    Pxk = tf.transpose(Pxk)
    print("here2")
    Pwk = np.transpose(Pwk)
    print("here3")
    Pwk = tf.transpose(Pwk)
    print("here4")

processes = []

for _ in range(3):
    p = Process(target=test_inverse, args=[], kwargs={})
    time.sleep(1)
    p.start()
    processes.append(p)

for process in processes:
    process.join()
    
### works perfectly fine
import time
time.sleep(5)
print("using ANY tensorflow function")
a = tf.math.add(2,5)

processes = []
for _ in range(3):
    p = Process(target=test_inverse, args=[], kwargs={})
    p.start()
    processes.append(p)
    
for process in processes:
    process.join()
```


### Relevant log output

```shell
2023-05-31 19:20:48.218419: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-31 19:20:48.286362: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-05-31 19:20:48.620505: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-05-31 19:20:48.620539: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-05-31 19:20:48.620543: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-05-31 19:20:50.183101: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2023-05-31 19:20:50.183138: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cedric-Z590-UD-AC): /proc/driver/nvidia/version does not exist
2023-05-31 19:20:50.183724: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
here1
here2
here3
here4
2023-05-31 19:20:51.221681: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2023-05-31 19:20:51.221727: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cedric-Z590-UD-AC): /proc/driver/nvidia/version does not exist
2023-05-31 19:20:51.222470: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
here1
here2
here3
here4
2023-05-31 19:20:52.223862: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2023-05-31 19:20:52.223908: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cedric-Z590-UD-AC): /proc/driver/nvidia/version does not exist
2023-05-31 19:20:52.224710: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
here1
here2
here3
here4
using ANY tensorflow function
2023-05-31 19:20:57.355866: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2023-05-31 19:20:57.355928: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cedric-Z590-UD-AC): /proc/driver/nvidia/version does not exist
2023-05-31 19:20:57.356675: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
here1
here1
here2
here3
here2
here3
here1
here2
here3
```
</details>