Why isn't the solar panel charging the battery?
You do need more than 12 volts to charge a 12 battery.  Also how do you know that your solar panel is producing even 12 volts?  Just because that is it's rating doesn't mean it always produces that value.  Do you have enough sunlight?  Have you measured the output of the panel? Can you provide more information on the charge controller? If you did provide 24 volts to the charge controller, how much voltage will it apply to the battery which is important because 24 volts is probably too much. Your panel is rated at 12 volts and 5 watts.  Thus, assuming it is getting sufficient sunlight, its maximum current is 0.42 amperes.  At that level, charging a 4.5 AH battery, would take a minimum of almost 11 hours.