What makes synapses stay "fixed"?
Synapses are held together by adhesion molecules, like cadherins and neuroligins/neurexins, so they aren't just loose adjacent membrane, they are securely anchored in place.

The adhesion molecules serve both a structural and regulatory role, being important for synapse formation and plasticity.

Missler, M., Südhof, T. C., &amp; Biederer, T. (2012). Synaptic cell adhesion. Cold Spring Harbor perspectives in biology, 4(4), a005694.