Do you 'own' Bitcoins purchased via an exchange?
Most exchanges are centralized services, operating in the way you've mentioned. So yes, the exchange owns the bitcoins, so there is significant counterparty risk involved with storing your coins on an exchange.