LM2776 gets damaged after some time
Since you don't know what is killing the LM2776, it's probably a good idea to bullet proof the design at this point. 

The absolute maximum ratings are being exceeded in some manner. If it's not from hot plugging, then ESD is a likely contender for failures in this device, most of them are fast, and usually happen when you don't have a scope plugged in. 

With devices that are hot plugged, there can be voltage spikes. Put a 5.2V zener and\or TVS on the output and input of the LM2776. The diodes across Vin and GND and the output would ensure that the voltage is kept within spec at all times. 

Then do your testing and handle it and see if the LM2776 still dies, if it does, then you need to look elsewhere and eliminate all possibilities of failure.  

To make sure the current is within range, put a series limiting resistor on output and input. (if you can on the output, the drop might be too much.)