If I bought bitcoin in the early days in some exchange for $10, would it have been lost if I kept it in the exchange? I just want to get some history perspective of Bitcoin.  Say in the early days when Bitcoin was $1 or $10, if I decided, &quot;ok, I may invest in US$3000 at the software startup I work for, but that US$3000 is likely to go to $0, so I will just invest $1000 in Bitcoin anyway, assuming it could be lost but not a big deal&quot;.
So what places was available for me to buy Bitcoin at that time?  If I bought it from BitInstant, Mtgox (I am not familiar and will need somebody who knows the history to answer this), then would that mean that $1000 could become $3 million, except it was all lost or stolen along the way during the BitInstant and Mtgox time?
But if I bought it and immediately transfer to my own wallet, and didn't forget the password, then it would have been ok?
You are correct.
Leaving your money in the hands of a third party such as an exchange means two things

so far as the Bitcoin network is concerned, you own no money at all.
when that business fails, your money is usually lost. You might get some back if there is a police investigation and/or some bankruptcy process - but you shouldn't rely on this.

Transferring the Bitcoin amount to the sole control of your own Bitcoin wallet means that your money is safe, so long as

you keep a copy of the private key(s) or seed-phrase somewhere safe, or make regular backups and remember any password.
your computer isn't hacked or infected.