ICSP or Pre-Programmed Microcontroller
If you have the space for a header, that's probably the best option, and the slightly staggered pins Majenko recommends is a good way to hold a connector in place without adding complexity.

But here's an alternative approach, for boards where you don't have space to fit a header, or the standard header doesn't accomplish all your programming/setup/board test tasks.

Given a small form factor PCB as here (33mm diameter) 


you can lay it out with test points (labelled on the back)


and create a test fixture for it (using Pogo Pins)


This requires some machining ability and at least a pillar drill (because the pogo pins must be exactly vertical). However if you're willing to risk a prototype PCB you can clamp it to a sheet of material and "spot" through the testpoint holes to get the pogo pins in exactly the right place. (I get 10 PCBs for a ridiculously low price from Itead, so losing one is no big deal).

With a lathe and mill, making the pillars and clamps to locate the PCB in exactly the right place is a doddle, but it's not too difficult with simpler tools.

Then you simply fit each PCB, program and test it. The loose connector normally goes to header pins on a TI Launchpad, used for programming this MPU (TI MSP430) via the SBW interface, but the same principle would apply to JTAG or other interfaces. (The brown cable at the top is a co-ax cable carrying a calibration signal to a frequency counter).


The polished mahogany base is optional; I just happen to prefer a steampunk style for my test fixtures.