Finder window pops up every time Time Machine makes a backup
I solved it by running

sudo defaults write com.apple.frameworks.diskimages auto-open-rw-root -bool false


Source: https://superuser.com/questions/584237/make-time-machine-stop-opening-a-new-window-when-backing-up