Alternative to backer rod and bond breaker tape for caulking
Backer rod should be compressed in a joint. If it doesn't stay in the joint on it's own, then you need to get a wider piece. They come in varying diameters and you should choose one slightly wider than your gap.