slight voltage reduction
I will try to answer.

Here is one low dropout linear regulator.

If you can see from data sheet, up to 1 A of load current, the dropout across the regulator will be less than 0.5 V.. It is the minimum drop which will happen. 

Hence even if input is 6V ish, still the output will be strong 5 V.

I would still recommend to look at linear regulator though you mention you are using a battery for its simplicity.

But be aware that there are circuits which can give battery better mileage but it depends on 


load current
Minimum voltage your circuit can operate
and price target or size target
noise over power line accepted etc


I will update my answer if you share more specific data.

A BAV720 diode for example drops easily about 0.7 V at a few 10s of mA providing you the required 5V.. Again it depends on how much tolerance your Circuit has for the fluctuation in the voltage.

Please also consider how long you need to run the circuit with batteries? What I'd the batteries drop down to 1.2 V? Then only 4.8 V will be available.

For 80 mA of load current, I would say the power lost across the regulator is negligible.

Edit 2:
Also consider running system down to 3 V.. You can find LEDs too for that range and you can also benefit from maximum juice out of the batteries. ATTiny can run down till 1.8 V /2.5 V.. Then best fit will be a buck regulator.