How could BTC be used and transacted on a daily basis without physical theft? I just feel a gaping hole in my logic, or maybe a duplicate question, so I'm prepared to quickly remove this question.

I was wondering how theft could be prevented in an economy that relies primarily on a cryptocurrency such as BTC. I own BTC, so I was wondering if someone believed that I had Bitcoin and wanted to rob me (perhaps at gunpoint), what would prevent him/her from exhausting all the money I have until he/she believes I have no money left? Again, this is supposing that this is in an economy that hypothetically gotten to the point that it relies primarily on Bitcoin and it would not be uncommon to find a BTC user by choosing at random. Would some sort of a third-party transaction-handler securely transact BTC in some way that would prevent theft? I can't seem to figure this one out.
Just to elaborate on the comment by Diego Basch:

Of course, Bitcoin has no solution for the fundamental problem that the threat of force can compel a person to do something they don't want to.  (Maybe that bug will be fixed in the next version of civilization.)  But street robbery should ultimately be less of a risk than you think.

Although it's possible to carry unlimited amounts of Bitcoin around with you (say, in a wallet app on your mobile phone), it's not envisioned that most people would handle their life savings that way.  Instead, just as with cash, you would carry with you only as much as you need for routine day-to-day spending.  The rest of your savings could be kept in "cold storage", with the private keys physically stored in a safe place that you access only when you need to replenish your wallet.  So a street robber couldn't steal all your coins, only those you had chosen to carry with you.

You could imagine other protections, too.  For example, a mobile wallet app would normally keep the keys encrypted and require a password to access them, so a thief who simply stole your phone wouldn't get your coins.  (They could try to crack the password, but by that time, you could use a backup copy of your private keys to move the coins to a new address.) 

Now, a robber could try to force you to enter that password.  But you could imagine an app with a "duress code", an alternate password that when entered causes the app to delete the keys (assuming you have a backup in a safe place).  It could even do something like silently delete most of the keys but leave a small amount available, so that the robber thinks that's all you had, takes it and leaves you alone.  Kind of the Bitcoin equivalent of keeping $2 in your wallet and your other $500 in your shoe.