Time Machine wants to backup all files after permission change — how to avoid?
In the end I just had to erase my time machine disks and start from scratch.
It seems to work fine now, but this is not how it should have worked.

So I would be looking into moving from Time Machine in favor of something more reliable and avoiding TM further at all costs.