How do nerve impulses travel so quickly?
There is a large gradient created by selective permeability of Na channels and as you're probably aware the speed of diffusion is proportional to the electrochemical gradient. 

Very small numbers of ions actually cross the membrane during depolarisation and subsequent AP events.