What is the final training result of asynchronous synchronous parallel distributed training？
@gogogwwb 
In sync training, all workers train over different slices of input data in sync, and aggregating gradients at each step. In async training, all workers are independently training over the input data and updating variables asynchronously. Typically sync training is supported via all-reduce and async through parameter server architecture. The final training result of async and sync parallel distributed training depends upon specific implementation, optimization, and training algorithm. Please refer to the [doc_1](https://www.tensorflow.org/guide/distributed_training#types_of_strategies) and [doc_2](https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras#choose_the_right_strategy) for your reference  and let us know if it helps.

Thank you ! Closing as stale. Please reopen if you'd like to work on this further.

 Thanks Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59740">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59740">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Support

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

tf2.8

### Custom Code

Yes

### OS Platform and Distribution

_No response_

### Mobile device

_No response_

### Python version

_No response_

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
When distributed training adopts asynchronous synchronous parallel, the parameters updated by each worker are inconsistent. Is the final result of distributed training the slowest parameter updated by workers?
```


### Standalone code to reproduce the issue

```shell
When distributed training adopts asynchronous synchronous parallel, the parameters updated by each worker are inconsistent. Is the final result of distributed training the slowest parameter updated by workers?
```


### Relevant log output

_No response_</details>