HMAC versus encrypted hash When encrypting a message with a symmetric key, it seems like the best practice is to first encrypt the message, then run the cipher text and key through HMAC, appending the result to the cipher text. The receiver can strip off the hash, HMAC the cipher text and key, and verify that the hashes match. This tells the receiver that the person who made the HMAC had the secret key, and that the message was not tampered with.

But why can't the sender instead hash the cipher text, then encrypt the hash with the secret key? The receiver would decrypt the hash with the same key, hash the cipher text, and verify that the hashes match. Wouldn't this give the receiver the same assurance, and isn't this how it's done in public key cryptography, when the sender wants to sign the cipher text?
$\newcommand{\concat}{\mathop{\Vert}}\newcommand{\AES}{\mathrm{AES}}$You are asking whether you can take a secure—specifically, IND-CPA—message cipher $E_k$, say AES256-CTR, and a fixed public collision-/preimage-/2nd-preimage-resistant hash function $H$, say SHA-256, and compose them in the form $$c \concat E_k\bigl(H(c)\bigr), \quad \text{where} \quad c = E_k(m)$$ to encrypt a message $m$ with a key $k$, and get a secure authenticated encryption scheme—specifically, IND-CCA3.  (I'm eliding details of nonces and input separation for the moment.  Figuring out how to fit them into this post is left as an exercise for the reader.)

This looks awfully close to encrypt-then-MAC, with the putative MAC $(k, c) \mapsto E_k\bigl(H(c)\bigr)$.  Unfortunately, on its own, that is not a secure MAC.  However, the input $c$ is an encrypted message, so that analysis doesn't end the discussion.

This also looks close to encrypt-and-MAC with the putative MAC $(k, m) \mapsto E_k\bigl(H(E_k(m))\bigr)$, which maybe is secure but it would take careful analysis to say, partly because it's not used in isolation—you also reveal $E_k(m)$ to the attacker.  I haven't done the analysis and I don't know anyone who has, so I won't say one way or another whether this is secure.

All that said, if you change this a little bit by using a secret choice of hash function $H_r$, giving $$c \concat E_k\bigl(H_r(c)\bigr), \quad \text{where} \quad c = E_k(m),$$ then you get the structure of, e.g., AES-GCM, which is a secure authenticated encryption scheme.

The family of functions $H_r$ in AES-GCM is called GHASH, and it isn't even preimage-resistant, let alone collision-resistant, to anyone who knows the index $r$—but $r$ is secret, perhaps derived alongside $k$ from some master key $k_0$.  Since its security requirements are more modest—in technical jargon, it must be an $\epsilon$-almost-universal hash family—GHASH is very fast and cheap to evaluate compared to SHA-256 or any other collision-resistant hash function, as long as you have fast binary polynomial multiplication and reduction.

(In software, you need special CPU instructions to do this fast in constant time, which are not widely consistently adopted in CPU instruction sets or standard programming languages, just like you need special CPU instructions that aren't widely consistently adopted to evaluate AES fast in constant time.  But you could use another stream cipher such as Salsa20 and another hash family such as Poly1305 which works in a prime field—these require no special CPU instructions to evaluate fast in constant time.  Note, however, that this is a little different from NaCl crypto_secretbox_xsalsa20poly1305, which is structured to also defend against a reforgery attack on Carter–Wegman many-time authenticators like AES-GCM uses.  For an even more variegated menagerie of authenticated encryption schemes, check out the CAESAR competition.)