Why is a 51% attack guaranteed to outpace the rest of the network's miners? Doesn't 51% hashing power give only a slightly higher chance? I thought hashing power is an expression of the probability of being the one to solve the cryptographic puzzle and placing a new block on the network.

If that's true, owning 51% of the hashing power should just give a slightly higher chance of mining a new block than 49% hashing power. However when I read articles explaining Bitcoin and mining, owning 51% of the hashing power in the network is presented as a guarantee to be able to outpace the rest of the network, to the point where an alternate chain can be created and funds double spent.

Given what I assume (could be wrong) is a marginally higher probability of solving the puzzle first at 51%, how does that make sense?
Couldn't someone who got a little lucky do that with 40% of the hashing power? How does 51% of the hashing power guarantee that one will be able to carry out an attack on the network?
With any probability distribution, a larger sample size will converge onto the expected probabilistic outcomes. 

So if you had 51% of mining power, there is a chance the other 49% of miners will outcompete you on a short time scale, but over longer time scales we’d expect you to win. 

Really, the threshold for this is just >50%, but a ‘51% attack’ has become the colloquial term for this. 

Note that a 1-2% of an advantage may not sound like much, but that’s the house favour on many casino games, and casinos generally seem to have no problem staying in business (despite sometimes paying out big to lucky players). Over long enough time scales, we reliably see a regression to the mean.