How should I handle invalid actions (when using REINFORCE)?
Just ignore the invalid moves.

For exploration it is likely that you won't just execute the move with the highest probability, but instead choose moves randomly based on the outputted probability. If you only punish illegal moves they will still retain some probability (however small) and therefore will be executed from time to time (however seldom). So you will always retain an agent which occasionally makes illegal moves.

To me it makes more sense to just set the probabilities of all illegal moves to zero and renormalise the output vector before you choose your move.