How best to recharge 3.6v Lithium Coin Cell batteries safely from small solar panels?
Yes you need to worry about over-charging them, and over-discharging them, and their temperature, and putting too much charging current into them, etc.  You need to build safety into your design.  You'll might take a look at something like this linear tech solar battery charger.  Notice how it has some safety features and a charge controller built into it.  It also has MPPT to which should help you get the most power out of your panels.

I know you want to use a 3V panel but then you'll need to boost the voltage to get high enough to charge them (if you're just using a single panel anyway).