5v and 3.3v from 2xAA and/or USB
My answer is all about losing the diode that is in series with the battery.

If you look at the workings of a booster you will see that there is a diode in the output stage such as here: -



Note also that there is a disable pin on many of the devices that are similar to the LT3467 - I'd be tempted to look at wiring USB 5V directly to the output "Vout" and then have some form of detector circuit that can shut-off/disbale the main part of the chip to conserve battery power. It needs some investigation of course.