Connectors for quick swapping PCB components
Autosplice (and possibly others) make solderless teminals like these:



See the datasheet here.  These can be pressed into the PCB with some gentle force if you don't have too many pins to connect at one time.  You'll need to solder the straight end into the main PCB.

Note that neither these headers nor the PCBs you press them into will have high cycle counts - The headers will flatten, and the hole plating on the daughter cards will be gouged by the headers.  Note also that the connection won't be as durable as a soldered link; don't subject these to lots of vibration or tension from a cable connection. If either of these problems are a showstopper, you'll need a standard connector with a soldered link.

However, for your "In case they fail" application, this should be a fine solution.  Sparkfun is out of stock, you can email the manufacturer at sales@autosplice.com or look at their distributors.