Why is a 51% attack guaranteed to outpace the rest of the network's miners? Doesn't 51% hashing power give only a slightly higher chance?
With any probability distribution, a larger sample size will converge onto the expected probabilistic outcomes. 

So if you had 51% of mining power, there is a chance the other 49% of miners will outcompete you on a short time scale, but over longer time scales we’d expect you to win. 

Really, the threshold for this is just >50%, but a ‘51% attack’ has become the colloquial term for this. 

Note that a 1-2% of an advantage may not sound like much, but that’s the house favour on many casino games, and casinos generally seem to have no problem staying in business (despite sometimes paying out big to lucky players). Over long enough time scales, we reliably see a regression to the mean.