Best workflow to mix several takes in VSE
Decide on your output framerate first (24, 25, 30, etc). Set that in the render panel and don't ever touch that value again or it will mess up the av sync.
In the timeline window, make sure AV-Sync is turn on. This will help you during preview playback.
Lay down the audio track first, and then lock that layer down with Shift+L.
Turn on the Draw Waveform for the audio track so you can see where the beats are.
Lay down all the video tracks you have, one on each 'layer', and make sure they are in sync with the audio from beginning to end.
Use the Multicam selector to switch between the video tracks.
If the video is in a compressed format with lots of P-frames, you won't be able to scrub through the video smoothly. Consider using the Proxy feature in Blender to generate a smaller, scrubbable version.


Below is a sample of how the setup might look. From the bottom, you see a locked audio track, 3 video-only track, and multicam selectors on the top.



Source: I've edited down 20GBs of footage down to a 4 minutes trailer with VSE before.