Time Machine started to backup files, it is still working and data is larger than actual disk data
This procedure helped to resolve the issue:


Turn off Time Machine and wait to completely stop
Erase Time Machine folder with all data (in my case a folder on NAS server but can be an USB disk).
Erase Time Machine configuration file on Mac: /Library/Preferences/com.apple.TimeMachine.plist
Setup Time Machine options. E.g. set exclude folders
Turn On Time Machine


I'm not sure about the conclusion, but suspect that .plist file was somehow corrupted.