Ensure snapshot files are finished writing to fs
Try with implementing a delayed assertion mechanism that allows for a certain amount of time before checking if all snapshot files have been fully written to the filesystem. This can be achieved by introducing a time delay or using a polling mechanism to periodically check for the existence of all the snapshot files.
Snapshot files may take a little time to all be written out to the filesystem on a heavily loaded system so need to allow for this before asserting that they are all there.

Fixes: https://github.com/tensorflow/tensorflow/issues/61116