Keep the voltage under 5V
You need a voltage regulator.  

Unfortunately, you've spec'ed batteries which are awfully close to the operating voltage of your system.  If you can use 5 batteries instead of 4 (giving you 6.something to 5V, instead of 5.something to 4V) , a low-dropout linear regulator will be a simple, easy solution.  The standard 7805 has too high a dropout for this purpose, but there are other pin compatible regulators; you'll want a TO-220 to dissipate the power that the Beagleboard can draw at full charge.

If you must use 4 batteries, you need to dissipate the excess voltage as heat through a MOSFET when the voltage is greater than 5V, and turn the MOSFET on if the voltage is less than 5V.  You are running slightly out of spec when you're below 5V, though I'm not sure what the absolute minimum voltage is for the board.

If you want the best solution possible, a buck-boost regulator would get you optimum efficiency for voltages slightly above and slightly below 5V.  You could even run it off a single battery, or from voltages much higher than 5V.  However, this is an expensive and complicated solution.  I'd recommend just using 5 batteries and an LDO.