I'd like to improve this design to support ICSP
A few things about your basic circuit before I address your specific questions.

i. Every IC needs a decoupling capacitor. For this circuit, that applies to the linear regulator and the microcontroller. For the microcontroller, as with most ICs, a 0.1uF capacitor between VCC and GND is typical. The capacitor should be as close as possible to the physical pins of the chip. The linear regulator takes a little more thought. Most regulator manufacturers will specify what kind of capacitors and what value to use on the input and output. This particular one does not, but they do provide a test circuit that should be good enough for your purpose:

ii. Every kind of linear regulator has a "dropout voltage" (VDO). The dropout voltage is the minimum difference in voltage between the input and the output. Your L78L05 has a dropout voltage of 1.7V. That means the minimum input must be: 
$$V_{INmin}=V_{OUT}+VDO=5V+1.7V=6.7V$$
Your 6V battery pack is less than 6.7V, which will not allow the regulator to regulate to 5V. Instead, you'll get something closer to 4V. That may not be a problem for this particular circuit, but you should be aware of that. There are plenty of linear regulators available with much lower than 1.7V VDOs.

Now on to your specific questions.


It normally doesn't matter where the microcontroller gets power during programming, as long as the power source is clean and stable. So either supply the power from your battery pack or from the programmer itself. But only choose one. If you leave the battery pack connected while enabling power from the programmer, they may conflict with each other. Whichever voltage source is slightly higher will try to push current into the other one, which won't make either one very happy. My recommendation would be to power the circuit off the battery and not the programmer. That way you don't have to keep removing the battery every time you program the board.


One possible complication here is if your circuit contains high-current components that might activate during programming. It is likely the programmer will not be able to supply enough current to keep the voltage stable if there is a lot of current draw from elsewhere. In a situation like that, it would be required to use your own power supply during programming.


You don't need to isolate the programmer when using the battery as long as the programmer's voltage supply is off. The only stipulation here is that the ground references of your circuit and the programmer are tied together. Otherwise, no special isolation is necessary.
If you do use the programmer's power, putting a diode on VOUT isn't a bad idea, but in may not be necessary. Unfortunately, the datasheet for the L78L05 does not give a absolute max voltage on the VOUT pin relative to the VIN pin. However, I have routinely applied external voltages to the output pins of unpowered linear regulators without a diode in between without problems. Without a specification in the datasheet, it's hard to know for sure. But I don't suspect you'll have a problem.
Yes, those are all valid concerns about adding a diode.
If you use the battery pack for power instead of the programmer, there are no more concerns.
Using resistors to isolate the other components from the programmer's pins is a common approach. However, you still need to be aware of how much current might be drawn from the programmer's pins. When a pin on the programmer goes high, the LED will light up. With a 100 Ohm resistor, a 2V LED may sink 30mA (\$(5V-2V)/100Ohm = 30mA\$). That's probably way more than the programmer's data pins can source. That's undoubtedly why the programmer is hanging up. 
Since you can't make the resistors any larger without making the LEDs dimmer, you can use digital buffers to remove the load on the programmer pins completely. Other types of components may need different types of isolation circuitry. But resistors and buffers cover most situations.








simulate this circuit &ndash; Schematic created using CircuitLab