Charging batteries in series when they are connected in parallel in the circuit
Your circuit doesn't do what you want it to - both batteries are permanently wired in parallel, as is the charger. The diode does nothing.

You probably don't want to charge them in series from USB anyway, because you would need at least 5.6V (1.4V/cell x 4 cells) but you only have 5.0V available. Charging in parallel can be done efficiently with a switch-mode current regulator, or just a resistor to limit current (and perhaps a blocking diode to prevent discharging into the USB port when not powered) if you don't mind a lower charge rate.