What are the most mergeable codecs? What is the best codecs for intermediate videos?
Mp4 with aac audio and h.264 video (mp4 isn't a codec as such, it's a container) isn't a good intermediate format. It is lossy, and the video is generally not all I-frame (meaning that to display a frame it references previous and following frames). h.264/aac/mp4 are great for delivery because they efficiently pack a lot of perceptual quality into a small file.

You're better off using an all-i-frame codec. Notable examples are Apple ProRes, Avid DNxHD and Cineform. These are all lossy, but minimally so. You can also use a lossless codec, like FFV1, but the speed and storage this requires might make it unworkable. Similarly you're best off using PCM (uncompressed) for your audio.

I'm not sure if that is going to affect the problem of the gaps. That might be an artefact of the way the video and audio are multiplexed.