Writing custom ReduceOp
Hi @parismic, to create an op, kindly follow this [guide](https://www.tensorflow.org/guide/create_op). Alternatively, you can also try creating a custom layer which is a common set of useful operations by following the instructions mentioned [here](https://www.tensorflow.org/tutorials/customization/custom_layers#implementing_custom_layers). Thank you!
 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60207">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60207">No</a>

I would like take a MirroredVariable, shape=(32,60),  with 3 replica and reduce is first as a difference of pairs and than L2 norm it. 

Is there a way of writing such a custom ReduceOp? 

https://github.com/tensorflow/tensorflow/blob/0db597d0d758aba578783b5bf46c889700a45085/tensorflow/python/distribute/reduce_util.py#L23-L47