Is it ok to keep rechargeable batteries in an unplugged charger?
I'd expect it to be OK in many cases - BUT no guarantees as it depends on the charger designer. If the charger was powered you'd hope most designers would place the cell in a neutral mode. You'd hope. Unpowered it is not hard to draw almost zero current, should the designer care and 'know their stuff'. 

An experiment:

Your question is easily answered in a specific case by an experiment.  

Place a battery in the unplugged charger with a small piece of paper between battery +ve nipple and +ve charger electrode so that the battery is isolated by the paper.
Set a multimeter to lowest current range and connect probes between charger +ve and battery +ve.
A well designed charger should produce at most &lt; 100 uA and it would be easy to get this &lt; 10 uA and not hard at all to get &lt; or &lt;&lt; 1 uA.

If smoke comes out of the meter while performing this test the charger is (or was) of an inferior design.

Lacking a low enough range ammeter, use a voltmeter - play to see what range gves highest reading. Now try eg a 1 megohm and 10 megohm resistor in series with bttery and meter and see how they compare.