VSE tracks get shifted
Two steps you haven't mentioned: every time you do anything that moves the frames you have to click "Refresh Sequencer" to be sure it is showing the correct frame. Also, if it loses sync while it is playing, change the Sync Mode from No Sync to AV-sync. It still isn't perfect, but it helps.