Using capacitor as a rechargeable battery
The simplest solution is to use a small 4.8V NiMh battery pack (just Google for examples). These are very common for hobby use to run the receiver and servos in radio-control gliders. These can generally be continuously trickle charged at 1/20 their "C" rating. For a very simple charger, you will need a higher voltage solar panel to get current into the battery and a series resistor to limit the charge current.

Example: Let's say you pick a common 600 mAh 4.8V pack. You should set your trickle charge no higher than 1/20 of this to prevent damage, or 30 mA. (Go read various NiMh battery specs.) Let's say you get a solar panel that outputs 9V. You will need (9 - 5) / .030 = 133 Ohm series resistor. 130 Ohms would be the closest standard value. The resistor would be dissipating heat at I^2*R or .030^2 * 130 = .117 Watts of heat, so you should use 1/2 Watt 130 Ohm resistor to give plenty of margin.

Hope that helps. -Vince