the feasibility of image processing techniques for physics based images
I think your intuition is correct, that it doesn't make sense to "clean" the matrices, if they are generated perfectly.  I agree with @SyedAliHamza that noise should be eliminated as much as possible in the simulation.

On the other hand, depending on what analysis you are trying to do, you may want your model to generalize better. In that case, it would be good to apply some data augmentation, meaning adding some noise or other types of augmentation to your matrices.