Why would you need to "catch up" in the 51% attack? There is a lot of information on the 51% attack where an attacker is able to branch the chain and "catch up" and build a new branch that is longer than the old branch. Other than requiring more than 50% of the network's computing power, the biggest thing that makes this attack infeasible is that the further behind an attacker is, the harder it is to catch up. Some places say that this means that a payment receiver can simply wait for a couple of blocks to go by and by that time is becomes infeasible for an attacker to catch up from that far behind.

But why would an attacker ever have to "catch up"? Couldn't an attacker with 51% of the network's power branch the chain as soon as a new block is released and work on that branch in secret? Then at any point the attacker could publish his branch, effectively reversing any transactions that occurred between the time he branched the chain and the time it is published to the network.
You're exactly right; there's no need to catch up if you don't start off behind.

As soon as I get 51% of the network's hashing power, I start building my fork in private, then buy something on the 'real' fork and allow the purchase to get as many confirmations as it needs.  It will be a lot, because I'll be buying something really expensive.  I take care not to include my transaction in the private fork.

Then, once the goods are shipped I publish my secret fork and get my coins back.

The Finney attack is only good for double spending when the recipient accepts unconfirmed transactions.