Why don't membrane proteins move?
Proteins can move around the membrane.
Most proteins do move within the membrane. The membrane is a liquid crystal and has fluid behaviour. Specifically, this is due to the membrane being in a gel-state. This gel state allows phase behaviour which means that the protein is able to move around on the surface. This results in an effect that is often referred to as the fluid mosaic model.
Proteins tend not to move out of the membrane.
The protein doesn't leave the membrane as a result of the transmembrane helix being very hydrophobic. This hydrophobicity and the hydrophobicity of the lipid tails means that they self-associate. A better way of describing it is that they fiercely dissociate from the water. A molecular dynamics simulation showed that even in simulations the membrane will readily self-assemble as a result of the hydrophobicity.
This is achieved by a few properties of the TMH sequence. There is a large amount of hydrophobic residues like leucine. At either end of the helix are large aromatic residues called the aromatic belt. Further is the electrostatic satisfaction offered by Gunnar von Heijne's famous positive inside rule and the recently identified negative-outside rule present in helices with evolutionary pressure to optimise anchorage.

(Image source: Baker et al., 2017)
Cymer et al. published a study showing the free energies associated with each part of the transmembrane protein helix (figure below). The overall ΔG for a transmembrane helix in the membrane is ~-12kcal mol−1. This means that the association of the helix in the membrane is typically spontaneous.

(Image source: Cymer et al., 2015)