Setting port in main loop causes ISR to starve
As pointed out in the comments, the problem is the read-modify-write sequence being interrupted, causing the instruction in the main loop inadvertently clearing PA4. 

When using a constant, GCC seems to optimize it down to a single sbi instruction. When using a variable, the port state is copied before the OR operation is performed. If the ISR fires between those instructions, the change from the ISR essentially gets lost.