The BOOTRST fuse was disabled, but bootloader code is still running after chip reset
You don't seem to have filled in all the interrupt vectors. I get this from your .bin file:

00000000 &lt;.sec1&gt;:
       0:       00 c0           rjmp    .+0             ;  0x2
       2:       00 00           nop
       4:       00 00           nop
       6:       00 00           nop
       8:       fc cf           rjmp    .-8             ;  0x2
       a:       fb cf           rjmp    .-10            ;  0x2
       c:       fa cf           rjmp    .-12            ;  0x2
       e:       f9 cf           rjmp    .-14            ;  0x2
      10:       f8 cf           rjmp    .-16            ;  0x2
      12:       f7 cf           rjmp    .-18            ;  0x2
      14:       f6 cf           rjmp    .-20            ;  0x2
      16:       f5 cf           rjmp    .-22            ;  0x2
      18:       ff ff           .word   0xffff  ; ????
      1a:       ff ff           .word   0xffff  ; ????
      1c:       ff ff           .word   0xffff  ; ????
      1e:       ff ff           .word   0xffff  ; ????
      20:       ff ff           .word   0xffff  ; ????
      22:       ff ff           .word   0xffff  ; ????
      24:       ff ff           .word   0xffff  ; ????
...
    7dfc:       ff ff           .word   0xffff  ; ????
    7dfe:       ff ff           .word   0xffff  ; ????
    7e00:       11 24           eor     r1, r1           &lt;--- bootloader
    7e02:       94 b7           in      r25, 0x34       ; 52
    7e04:       14 be           out     0x34, r1        ; 52
    7e06:       89 2f           mov     r24, r25
    7e08:       8d 70           andi    r24, 0x0D       ; 13
    7e0a:       11 f0           breq    .+4             ;  0x7e10
    7e0c:       89 2f           mov     r24, r25
    7e0e:       de d0           rcall   .+444           ;  0x7fcc
    7e10:       85 e0           ldi     r24, 0x05       ; 5


Now I can't quite see the relationship between the brown-out and those higher interrupts, but if one of them happened to fire I think it would waddle its way past all those 0xFF instructions and hit the bootloader.

The Atmega328P has 26 interrupt vectors (including Reset).