Powering a very low current application
Probably your best bet is going to be to use a linear regulator of the low drop-out variety. These are usually called "LDO's". The parameter you need to scrutinize is the quiescent current, also sometimes given as "Iq". This is the current consumed by the regulator when the output current is zero. Some LDO's have very low Iq of less than 1uA.

Some switching regulators also have low Iq, but the best LDO's will have lower Iq than the best switching regulator. Because of your usage profile, the low Iq will win over the higher efficiency during the brief instant when your devices are not sleeping.