What, exactly, does the REINFORCE update equation mean?
The loss function you are looking for is cross entropy loss. The 'label' that you use is the action you took at the time point you are updating for.