Do I need to provide voltages higher than the battery's present voltage to further charge it?
Well, to push in charges into anything, you need a voltage difference. So, yes.

Generally: You usually don't charge batteries just by connecting them to an uncontrolled voltage source. The correct method for charging a battery depends fully on its type, its current charge status and usage scenario. But physically, whenever a battery is charged, the voltage applied externally must be higher than the battery voltage. Otherwise, you'd do nothing (external potential = battery potential, i.e. no current flows), or discharge it (external potential &lt; battery potential, i.e. the battery provides your external "charger" with power, not the other way around).

This is a very basic consideration about what voltage is. Maybe you want to brush up your very electronics basics before charging batteries, since depending on their type, they can overheat, gas out (explosive hydrogen, for example), or simply catch fire or explode.