Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?
After thinking about this, I've realized that $Q(s,a)$ relies on the action and thus cannot be pulled out of the sum in the same way $B(s)$ can. I'm leaving this up for anyone interested in the same thing.