Why is the ledger public? To me, banking is not a public thing. I don't want people to know what's in my bank account at what time, or who I sent money to. 

I know that I can make a new address for each transaction, as addresses are 'free'. But this isn't the case for say, a coffee shop, which would want only a single address for people to send money to (usually in the form of a QR code printout hanging on the wall). I can effectively track the incoming and outgoing cash flow of a coffee shop by just tracing the blockchain, can't I? If I were a rival shop, this would be very useful to me.
When Bitcoin was invented, Blockchain analysis was not as advanced as it was today and pseudonymous transactions were believed to be closer to anonymous than they are with Bitcoin today. Under the current Bitcoin system, privacy can be greatly improved by avoiding fiat entry/exit ramps connected to your identity and by avoiding reusing BTC addresses. 

Yes Bitcoin transactions for the coffee shop you describe can easily be tracked. As you mentioned, stealth addresses can bring privacy improvements but not ideal for every situation. Once Confidential Transactions are added to Bitcoin (https://bitcoinmagazine.com/articles/bitcoin-privacy-confidential-transactions-feature-can-fix-some-of-coinjoin-s-problems-1457703275), privacy will improve but still not meet the level of the top 2 decentralized solutions solutions described below:


Ring signatures exist already with Monero and Ring Confidential Transaction (RingCT) is one of the most promising areas of privacy development: https://eprint.iacr.org/2015/1098.pdf
ZCash is another leading alternative if the market is satisfied with its trusted setup solution and developer mining reward share: https://blog.okturtles.com/2016/03/the-zcash-catch/