How can I maintain a fix voltage until my battery is almost dead?
A battery's voltage naturally drops as it discharges. To get a stable voltage out of it, you want a voltage regulator. Voltage regulators come in two flavors, linear and switching. If you want the output voltage of the regulator to ever be higher than the battery voltage, you need a switching regulator. You'll also get higher efficiency that way, so your battery should last longer.

Now, in your specific case, you really need to look at the input voltage spec for your Arduino board. If it says it can handle 6-9 volts, or something like that, then an external regulator's not going to help. The board is already doing the best it can. To make it last longer, you'd need to parallel more batteries.