Slow charging a li-po battery
Really the solution here is to just get a charger IC such as the one from TI or Maxim.
The problem is that you cannot usually charge a battery with Constant Voltage. Lithium Ion/Polymer batteries usually require Constant Current (CC) charging at first, and after the battery voltage reaches a certain point, then you can apply Constant Voltage. So, unless your batteries are not deeply discharged, you'll need constant current.

Safety is very important with Lithium batteries, so it's best to take the safe route.

For you i'd recommend picking up some sparkfun boards that can do the charging, like this one.