How do exchanges store and transfer bitcoin? When I buy bitcoin on an exchange I instantly get them and can withdraw my coins 5 seconds after the transaction. This means the exchange must record your account balance in some database independent of a bitcoin wallet (otherwise buying the coins would take ~10 minutes) and when I withdraw them it is sending a request to withdraw X bitcoin from the exchanges wallet.

However, when I check the wallet that BTC is coming from on blockchain.info etc the source wallet is always different, and I'm unable to see what wallet's balance over time.

How are exchanges storing bitcoin so you can buy/sell on the exchange instantly, and how are they managing that system for withdrawls?
It is important to keep in mind that addresses are not the same as wallets. A wallet contains multiple addresses, not each address is a wallet.

Exchanges maintain their own wallet with thousands of addresses and thousands of coins. When you deposit Bitcoin at an exchange, they will give you an address from their wallet, and you are sending to their wallet. The Bitcoin is now theirs. When you withdraw from the exchange, they send Bitcoin from their wallet to the address that you provide. When they send, their wallet uses some coin selection algorithm to select which coins it will spend to send to you. Since an exchange processes thousands of deposits and withdrawals per day and since the coin selection algorithm may include some randomness, the addresses associated with the coins being spent for each withdraw that you make will likely be different.