Non deterministic random numbers generated in Eager and Graph mode
@mergian Thanks for reporting this issue.

I was able to reproduce this issue in TF 2.11 and TF Nightly 2.13.0-dev20230209 . Please find the gist [here](https://colab.research.google.com/gist/pjpratik/ebeaa08d55c8797134f250ffa20606b4/59635.ipynb).

@SuryanarayanaY Could you please look into this. Thanks. Hi @mergian ,
Did you have any plan to fix this ? Please feel free to create a PR for this. Thanks! Hey @SuryanarayanaY

I unfortunately don't have a good idea. To my understanding the problem is as follows:

In eager mode, TF checks if a random operation is "unique" using the local seed. So TF allocates the local seed object `PhiloxRandom` once for `123` in my example, and uses the same object for both calls.

Incontrast, in graph mode TF creates local seed objects for each occurance of `tf.random.uniform(...)` within the graph, irrespectful of the seed argument.


This said: Syntactically I think eager mode is actually correct, as the user specified the identical local seed, and therefore TF should use the same local seed object for both calls. The problem however might come then when applying distribution strategies, where the first and second call to `tf.random.uniform(..., seed=123)` might not be executed on the same device, so sharing the same local seed object would not be possible or at least, difficult to implement.

The other way around, I think it's not possible for eager mode to identify that the two calls are not the same.


A "quick fix" would be to document this behavior and make users aware that they should use different local seeds in their random calls to have correct behavior.


Other solutions would likely require API changes. E.g., I've looked into how PyTorch is doing their random numbers ( https://pytorch.org/docs/stable/generated/torch.rand.html ). The user can either provide a `torch.Generator` object, which is a local seed object, or otherwise it will fall back to use the global seed object. I think this approach was not incorporated into TF to enable local random states for distributed computations. For distribution strategies, it is already documented as stated below.

> Do not use [tf.compat.v1.Session](https://www.tensorflow.org/api_docs/python/tf/compat/v1/Session) and [tf.distribute.experimental.ParameterServerStrategy](https://www.tensorflow.org/api_docs/python/tf/distribute/experimental/ParameterServerStrategy), which can introduce nondeterminism. Besides ops (including [tf.data](https://www.tensorflow.org/api_docs/python/tf/data) ops), these are the only known potential sources of nondeterminism within TensorFlow, (if you find more, please file an issue). Note that [tf.compat.v1.Session](https://www.tensorflow.org/api_docs/python/tf/compat/v1/Session) is required to use the TF1 API, so determinism cannot be guaranteed when using the TF1 API.

Additional details are mentioned here https://www.tensorflow.org/api_docs/python/tf/config/experimental/enable_op_determinism cc @qlzh727  Please see [the RNG guide](https://www.tensorflow.org/guide/random_numbers). Thanks for posting the two guides. I agree that using the newer variants with Generator or Stateless should not produce this problem. Still: as long as `tf.random.uniform` is not deprecated, I think the bug should be mentioned in the corresponding documentation: https://www.tensorflow.org/api_docs/python/tf/random/uniform

Also in the `tf.random.uniform` page it is not mentioned that it is strongly discouraged to use these.
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

No

### Source

binary

### Tensorflow Version

2.11.0

### Custom Code

No

### OS Platform and Distribution

Linux CentOS

### Mobile device

_No response_

### Python version

3.7

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
Executing the identical Keras model in eager and graph mode produce different results when the identical seed is passed to `tf.random.uniform`. I created a small example that just generates 2 random outputs. In one case, I use different values for seed, in the other I use identical seeds.

As you can see in the output, in the case where the seeds differ, I get the same result using `model(data)` and `model.predict(data)`. However, when my seeds are identical, then I get other results in eager mode.

I tried to pinpoint the source for this, and I think I found it. I modified the PhiloxRandom class to do some output whenever a function is called. It shows the pointer of `this`, which function gets called and also the `key[0:2]` and `columns[0:4]` values. `key` gets initialized through the global seed `tf.random.set_seed(...)` and `columns[2:4]` through the `seed` value passed to `tf.random.uniform`.

What we see is, that in the case where the seed differs, TensorFlow creates two instances of `PhiloxRandom(...)`, one with 0x7C (==124) and one with 0x7B (==123) as seed. The same happens in graph mode. I think this is the expected behavior.

In the case where the seeds are identical, we see that in eager mode only one instance of PhiloxRandom is created, and it gets called twice. So it uses an accumulated seed in the second `tf.random.uniform(...)`, which is wrong. My assumption is that TensorFlow in eager mode caches the operator instance and just reuses it when the "identical" layer gets called. However, in this particular case this causes undeterministic results, which differ from the graph mode.
```


### Standalone code to reproduce the issue

```shell
import tensorflow as tf
import numpy as np

# see: https://www.tensorflow.org/api_docs/python/tf/config/experimental/enable_op_determinism
tf.keras.utils.set_random_seed(1)
tf.config.experimental.enable_op_determinism()

def create(seed_a, seed_b):
    inp = tf.keras.Input(shape=(5,))
    x = inp / inp # == 1.0
    a = tf.random.uniform(tf.shape(x), dtype=x.dtype, minval=-5, maxval=5, seed=seed_a) * x
    b = tf.random.uniform(tf.shape(x), dtype=x.dtype, minval=-5, maxval=5, seed=seed_b) * x
    return tf.keras.Model(inputs=[inp], outputs=[a, b])

def compare(seed_a, seed_b):
    print(f"## RUNNING {seed_a} {seed_b} ##")
    data = np.ones((1, 5), np.float32)

    model = create(seed_a, seed_b)

    eager = model(data)
    graph = model.predict(data)

    for i, (a, b) in enumerate(zip(eager, graph)):
        print(f'#{i}, A: {a.numpy()}, B: {b}')

tf.random.set_seed(314159)
compare(123, 124)

tf.random.set_seed(314159)
compare(123, 123)
```


### Relevant log output

```shell
# EXECUTION RESULTS --------------------------------------------------------- #

## RUNNING 123 124 ##
#0, A: [[ 0.64956665  4.783145   -4.816675    4.7801285   1.6770649 ]], B: [[ 0.64956665  4.783145   -4.816675    4.7801285   1.6770649 ]]
#1, A: [[ 1.321938   4.436612   2.8036633 -2.5721383 -3.595649 ]], B: [[ 1.321938   4.436612   2.8036633 -2.5721383 -3.595649 ]]

## RUNNING 123 123 ##
#0, A: [[ 2.2466993   2.8111506   1.4583216  -0.34343958  2.204914  ]], B: [[ 0.64956665  4.783145   -4.816675    4.7801285   1.6770649 ]]
#1, A: [[ 0.64956665  4.783145   -4.816675    4.7801285   1.6770649 ]], B: [[ 0.64956665  4.783145   -4.816675    4.7801285   1.6770649 ]]


# /EXECUTION RESULTS -------------------------------------------------------- #

# DEBUG --------------------------------------------------------------------- #

## RUNNING 123 124 ##

### EAGER
0x7fffb27e8170 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007C 00000000
0x4101a50 Skip(1280)
0x7fffb27e86f0 Skip(0)
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007C 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 5150EBBA 4EF8C9E4 4D63E30B 919F139E
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007C 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / BC11F9C7 A411A7B1 99682563 B50F8A71
0x7fffb27e8170 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x412a130 Skip(1280)
0x7fffb27e86f0 Skip(0)
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 5EC85080 05FD3969 A08258B8 61FD2F86
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 4DD57768 F92C9439 A139802D 55996F6F

### GRAPH
0x7fffb27e9890 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007C 00000000
0x7fffb27e9890 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x441f8b0 Skip(1280)
0x7f8e1fffda30 Skip(0)
0x7f8e1fffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7f8e1fffda30 OperatorEND(): 8FF812B0 96A522AD / 5EC85080 05FD3969 A08258B8 61FD2F86
0x7f8e1fffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007B 00000000
0x7f8e1fffda30 OperatorEND(): 8FF812B0 96A522AD / 4DD57768 F92C9439 A139802D 55996F6F
0x4454e30 Skip(1280)
0x7f8dfbffda30 Skip(0)
0x7f8dfbffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007C 00000000
0x7f8dfbffda30 OperatorEND(): 8FF812B0 96A522AD / 5150EBBA 4EF8C9E4 4D63E30B 919F139E
0x7f8dfbffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007C 00000000
0x7f8dfbffda30 OperatorEND(): 8FF812B0 96A522AD / BC11F9C7 A411A7B1 99682563 B50F8A71


## RUNNING 123 123 ##
### EAGER
0x7fffb27e8170 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x4167510 Skip(1280)
0x7fffb27e86f0 Skip(0)
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 5EC85080 05FD3969 A08258B8 61FD2F86
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 4DD57768 F92C9439 A139802D 55996F6F
0x4167510 Skip(1280)
0x7fffb27e86f0 Skip(0)
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000500 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / C7DCC1FC 4C63FB94 2352AAA1 D83B9A9E
0x7fffb27e86f0 OperatorBEGIN(): 0004CB2F 00000000 / 00000501 00000000 0000007B 00000000
0x7fffb27e86f0 OperatorEND(): 8FF812B0 96A522AD / 8DDC3910 CEB432E2 6CF2E131 6770FC94

### GRAPH
0x7fffb27e9890 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7fffb27e9890 PhiloxRandom(uint64_t, uint64_t): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x440ed40 Skip(1280)
0x7f8e1fffda30 Skip(0)
0x7f8e1fffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7f8e1fffda30 OperatorEND(): 8FF812B0 96A522AD / 5EC85080 05FD3969 A08258B8 61FD2F86
0x7f8e1fffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007B 00000000
0x7f8e1fffda30 OperatorEND(): 8FF812B0 96A522AD / 4DD57768 F92C9439 A139802D 55996F6F
0x4500200 Skip(1280)
0x7f8dfbffda30 Skip(0)
0x7f8dfbffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000000 00000000 0000007B 00000000
0x7f8dfbffda30 OperatorEND(): 8FF812B0 96A522AD / 5EC85080 05FD3969 A08258B8 61FD2F86
0x7f8dfbffda30 OperatorBEGIN(): 0004CB2F 00000000 / 00000001 00000000 0000007B 00000000
0x7f8dfbffda30 OperatorEND(): 8FF812B0 96A522AD / 4DD57768 F92C9439 A139802D 55996F6F

# /DEBUG -------------------------------------------------------------------- #
```
</details>