Need help finding a solution to recharging old Li-ion batteries, maybe a power supply
As your working with what is likely single cell li-ion, and need to just make sure you get them to a chargable state, set your power supply to about 2.7V, and fit a 2.7K resistor in series, this will very slowly trickle charge it back up,

If it does not increase in voltage after a long while, the battery leaks to a point it is not really usable. 

Normal lithium safety still applies here, charge them somewhere that if it exploded into fire, it would not harm you or others, the proper way would be something to warn you if it became warm, but at a slow trickle charge, if it became warm, its already breaking down internally.

The lipo chargers for RC vehicles are a suitable option, but many have the same issue that they will not respond when flatter than they expect.