Why not just encrypt the message and the hash of the message to achieve cryptographic integrity? I've always seen MACs as a way to provide integrity and authentication. You would just append it to some message in clear.

But this is clearly not the case when you're doing Encrypt Then Mac. When you do that you provide authentication to something that already has authentication (to decrypt we need to know the key). So why use a MAC when we can use a hash instead: $E(m | h(m))$? (i.e., encrypt the message appended to a hash of the message)
I'll take this in parts:


  But this is clearly not the case when you're doing Encrypt Then Mac. When you do that you provide authentication to something that already has authentication (to decrypt we need to know the key).


Encryption, by itself, does not provide authentication.


  why use a MAC when we can use a hash instead: E(m|h(m))?


Here is a simple counter example to your proposal. Say I know that m="Transfer \$1000 to Mikeazo", and you use a stream cipher (or stream like mode) to encrypt that and the hash of the message. I know how to compute h(Transfer $9999 to Mikeazo). I also know what bits to flip in the encryption to get everything to check out okay.

Now, you may be thinking that this attack is only valid since I assumed a stream cipher and how about other modes (e.g., CBC). Well, here are the details on an attack with CBC. 

When we design cryptographic primitives, we want them to be as secure as possible, and even provably secure (given a reasonable set of assumptions) if possible. That is why we do things like Encrypt-then-MAC as opposed to your proposal. Encrypt-then-MAC is provably secure.