tf.distribute.MultiWorkerMirroredStrategy does not replicate tf.Variable among workers
@pfaeffli 
I tried to replicate the issue on the virtual linux machine using TFv2.12  but facing a different error. Please find the below screenshot for reference:
![image](https://user-images.githubusercontent.com/111861663/228419639-f6079216-4187-4ac0-83cd-ead77797fa8b.png)
Could you please provide detailed steps to replicate the issue reported here ?

Thank you ! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60128">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60128">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.9

### Custom Code

Yes

### OS Platform and Distribution

tensorflow/tensorflow:2.11.0

### Mobile device

_No response_

### Python version

3.8.10

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
A variable is created with `synchronization=tf.VariableSynchronization.ON_WRITE`. But if `.assign(1)` is called the value is not distributed among workers.

Expected behavior: the variable is set to `1` on all workers.

There might be a lack of understanding on my side on how the distribution is working.
```


### Standalone code to reproduce the issue

```shell
`src/minimal-example.py`:


#!/usr/bin/env python3
# COPYRIGHT HSLU 2023 PFAEFFLI, ALL RIGHTS RESERVED
import json
import logging
import os
import time
from pathlib import Path

import tensorflow as tf
os.environ[
    "CUDA_VISIBLE_DEVICES"
] = "-1"  # Prevent that any GPUs are allocated unintendendly


def _is_chief(task_type, task_id, cluster_spec):
    return (
        task_type is None
        or task_type == "chief"
        or (
            task_type == "worker"
            and task_id == 0
            and "chief" not in cluster_spec.as_dict()
        )
    )

def train():
    if "WORKER" not in os.environ:
        logging.warning("Missing env 'WORKER'")
        wname = "UNK"
    wname = os.environ["WORKER"]

    # Logger
    FORMAT = "%(asctime)s - " + wname + " %(message)s"
    logging.basicConfig(format=FORMAT)
    logger = logging.getLogger(wname)
    logger.setLevel(logging.INFO)
    logger.info(f"{tf.version.GIT_VERSION}, {tf.version.VERSION}")

    # Load arguments
    tf_config = json.loads(os.environ["TF_CONFIG"])
    cluster_spec = tf.train.ClusterSpec(tf_config["cluster"])
    # Debug info to check if tf_config is set correctly
    logger.info(f"{wname}: TF_CONFIG {os.environ['TF_CONFIG']}")

    # Prepare model
    strategy = tf.distribute.MultiWorkerMirroredStrategy()
    with strategy.scope():
        dist_is_ready = tf.Variable(
            initial_value=tf.constant(0, dtype=tf.dtypes.int64),
            name="dist_is_ready",
            synchronization=tf.VariableSynchronization.ON_WRITE,
            aggregation=tf.VariableAggregation.SUM,
        )

    logger.info(f"{wname}: Replicas in sync: {strategy.num_replicas_in_sync}")
    logger.info(f"{wname}: {dist_is_ready}")

    task_type, task_id = (
        strategy.cluster_resolver.task_type,
        strategy.cluster_resolver.task_id,
    )
    if _is_chief(task_type, task_id, cluster_spec):
        logger.info(f"{wname}: Assign ready")
        dist_is_ready.assign(1)
    else:
        while dist_is_ready.numpy() < 1:
            logger.error(f"{wname}: dist_is_ready is {dist_is_ready.numpy()}...")
            logger.error(f"{wname}: Wait for chief...")
            time.sleep(5)

    logger.info(f"{wname}: All done. dist_is_ready is {dist_is_ready.numpy()}.")


if __name__ == "__main__":
    train()
```
`docker/Dockerfile`

```
FROM tensorflow/tensorflow

RUN pip install --quiet --upgrade --pre tf-nightly
RUN mkdir -p /app/log

COPY ./src/* /app/

WORKDIR /app
```

`docker-compose.yml`

```
services:
  worker0:
    image: local/worker
    build:
      dockerfile: ./docker/Dockerfile
      context: ./
    command: ["./minimal-example.py"]
    networks:
      - cluster
    environment:
      TF_CONFIG: '{"cluster": {"worker": ["worker0:30000", "worker1:30000"]}, "task":
        {"type": "worker", "index": 0}}'
      TF_LOGDIR: /app/log
      WORKER: worker0
    volumes:
      - ${PWD}/log/worker0:/app/log

  worker1:
    image: local/worker
    build:
      dockerfile: ./docker/Dockerfile
      context: ./
    command: ["./minimal-example.py"]
    networks:
      - cluster
    environment:
      TF_CONFIG: '{"cluster": {"worker": ["worker0:30000", "worker1:30000"]}, "task":
        {"type": "worker", "index": 1}}'
      TF_LOGDIR: /app/log
      WORKER: worker1
    volumes:
      - ${PWD}/log/worker1:/app/log

networks:
  cluster: {}
```

Start with
`docker compose build && docker compose up`
```


### Relevant log output

```shell
distributed-learning-worker1-1  | 2023-03-27 07:52:46,080 - worker1 worker1: dist_is_ready is 0...
distributed-learning-worker1-1  | 2023-03-27 07:52:46,080 - worker1 worker1: Wait for chief...
distributed-learning-worker0-1  | 2023-03-27 07:52:46,082 - worker0 worker0: All done. dist_is_ready is 1.
distributed-learning-worker0-1 exited with code 0
distributed-learning-worker1-1  | 2023-03-27 07:52:51,087 - worker1 worker1: dist_is_ready is 0...
distributed-learning-worker1-1  | 2023-03-27 07:52:51,087 - worker1 worker1: Wait for chief...
```
```
</details>