Numerical precision of tf.math.cumsum operation on float32 vs float64
Hi **@sapphire008** ,
 I was able to reproduce this issue on Colab using TF v2.15, TF-nightly, Please find the [gist](https://colab.research.google.com/gist/Venkat6871/aa5298e814ce7c1652b57c9a3a1d6ea2/62687_2-15-v.ipynb), [gist1](https://colab.research.google.com/gist/Venkat6871/62dc5781e4c6aa38162ed7fec3dc668c/62687_nightly-v.ipynb) here for reference. 

Thank you!
### Issue type

Performance

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

source

### TensorFlow version

2.15.0

### Custom code

Yes

### OS platform and distribution

Linux, MacOS M1 Chip

### Mobile device

_No response_

### Python version

3.10.12

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

I would like to report an interesting observation regarding numerical precision of cumsum operations implemented by Tensorflow (`tensorflow==2.15.0`), PyTorch (`torch==2.1.0`), and NumPy (`numpy==1.26.2`).

I have tested the example script (pasted in the Standalone code to reproduce the issue)  that compares the cumulative sum of  a large array of numbers. On MacOS Sonoma 14.1.1, MacBook Pro 2021 Apple M1 Pro machine, I have obtained the following results:

```text
Float32 results:
torch vs. numpy float32: 4.625
tf vs. numpy float32: 0.0


Float64 results:
torch vs. numpy float64: 0.0
tf vs. numpy float64: 0.0
torch f64 vs. f32: 0.007812499505234882
tf f64 vs. f32: 4.619280484243063
np f64 vs. f32: 4.619280484243063
```

On Colab with CPU (Linux OS) instance using the same versions of the library, I have

```text
Float32 results:
torch vs. numpy float32: 5.15625
tf vs. numpy float32: 0.0


Float64 results:
torch vs. numpy float64: 0.0
tf vs. numpy float64: 0.0
torch f64 vs. f32: 0.007812497526174411
tf f64 vs. f32: 5.155536882346496
np f64 vs. f32: 5.155536882346496
```

Both are saying, on CPU, Tensorflow obtains the same result as NumPy, which loses certain degree of numerical precision in the result of cumsum with float32 inputs. PyTorch, however, does not lose much of the precision.

However, if I run the same code on Colab with GPU (with Linux OS), using identical versions of the library, I have:

```text
Float32 results:
torch vs. numpy float32: 7.328125
tf vs. numpy float32: 7.34375


Float64 results:
torch vs. numpy float64: 0.0
tf vs. numpy float64: 1.4551915228366852e-10
torch f64 vs. f32: 0.007812497817212716
tf f64 vs. f32: 0.1570705088088289
np f64 vs. f32: 7.322426828060998
```

This shows that Tensorflow does not lose much precision on GPU, though slightly more than PyTorch. NumPy still loses precision (since it still uses CPU).

This is an interesting observation. The behavior of PyTorch's cumsum seems more desirable, since it can significantly cut down memory during inference time while retaining certain numerical precision.

### Standalone code to reproduce the issue

Link to [Colab](https://colab.research.google.com/drive/1WBd0FhYacxmak2gVS4GNctRS-IxeJc7M?usp=sharing)

```shell

import numpy as np
import tensorflow as tf
import torch

# float32
X = np.random.rand(16, 500000).astype(np.float32)
np_cumsum_fp32 = np.cumsum(X, axis=1)
tf_cumsum_fp32 = tf.cumsum(tf.constant(X, tf.float32), axis=1)
torch_cumsum_fp32 = torch.cumsum(torch.tensor(X, dtype=torch.float32), 1)

print("Float32 results:")
print("torch vs. numpy float32:", np.abs(torch_cumsum_fp32.numpy() - np_cumsum_fp32).max())
print("tf vs. numpy float32:", np.abs(tf_cumsum_fp32.numpy() - np_cumsum_fp32).max())

# Casting the original value to float64
X = X.astype(np.float64) # same data!
np_cumsum_fp64 = np.cumsum(X, axis=1)
tf_cumsum_fp64 = tf.cumsum(tf.constant(X, tf.float64), axis=1)
torch_cumsum_fp64 = torch.cumsum(torch.tensor(X, dtype=torch.float64), 1)
print("\n")
print("Float64 results:")
print("torch vs. numpy float64:", np.abs(torch_cumsum_fp64.numpy() - np_cumsum_fp64).max())
print("tf vs. numpy float64:", np.abs(tf_cumsum_fp64.numpy() - np_cumsum_fp64).max())
print("torch f64 vs. f32:", np.abs((torch_cumsum_fp32 - torch_cumsum_fp64).numpy()).max())
print("tf f64 vs. f32:", np.abs(tf_cumsum_fp32.numpy() - tf_cumsum_fp64.numpy()).max())
print("np f64 vs. f32:", np.abs(np_cumsum_fp32 - np_cumsum_fp64).max())
```


### Relevant log output

_No response_