Downsides of not using deterministic ECDSA signatures for blockchain validation?
No, you'll be fine (most likely).

DSA (the base of ECDSA) has been around since at least 1993 (when FIPS 186 was released). Much like with ECDSA you also need a $k$-value for DSA for each signature. From what I know, RFC 6979 (August 2014, first draft in march 2011) was the first general standard and Ed25519 (special case of EdDSA now, september 2011 (PDF)) the first specific DSA variant to use deterministic $k$-generation. So essentially people used random $k$ values for nearly 20 years and issues came up only when people failed to ensure the randomness of the value (by re-using it or using a bad RNG). However this does not apply to you as your HSM will take care of that. So you'll be just as fine as all the people were before we had deterministic (EC)DSA.

The only drawbacks are essentially those that you listed in your question, the first two of which don't matter if you trust your HSM and the last one you can't do anything about.

Also note that a deterministic (EC)DSA signature is indistinguishable from a non-deterministic one except if you have two signatures of the same data.