Bitcoins Anonymity - Do Exchanges Break It?
could authorities subpoena bitcoin exchanges for information regarding any accounts of anyone who bought bitcoin and sent the BTC to that public key?

Yes.

It seems the exchange of USD for BTC almost requires bank account information.

No.
Non-KYC exchanges: Bisq and HodlHodl