Why are policy gradients on-policy?
The Policy Gradient theorem states that the gradient of the expected reward is equal to the expectation of the log probability of the current policy multiplied by the reward. Notice that to compute the integral of the expectation we can use Monte Carlo method. For this you would need sampled trajectories. Have you looked at a derivation? The whole theorem comes from mathematical derivations. Check in that page the Assumptions and the Likelihood Ratios (REINFORCE) sections. Hope it helps!