Theft deterrents (in addition to locking up a bike)
In addition to locking your bike securely with a good lock, there are several strategies you can use:
Location
Where you lock up can be nearly as important as how well you're locked up. If you're in a public, well-lit area, a thief might think twice about stealing a bike. A rack in an isolated, dark area may look more attractive to steal -- there aren't as many potential witnesses.
Beater bikes
While some may not have a choice in this matter, if you know you'll have to lock up in a sketchy neighborhood fairly often, you might consider getting a cheap, used mountain bike if possible. Save the flashy carbon expensive-looking bike for club rides and longer road rides.
Take the bike with you
There may be situations where you don't have to lock the bike up. Many supermarkets and big box stores will let you take the bike with you, if the aisles are roomy. Take care to stay out of the way of other customers. If asked to take the bike outside, you can either comply with the request or take your business elsewhere, but be nice about it either way.
Folding bikes are great for this, especially when you can fit the bike into a shopping cart.
Uglify your bike
I've heard it said that you just need to lock up your bike better than the next bike in the rack. That may or may not be true, but consider making your bike uglier than the bike next to it. A terrible rattle-can paint job (i.e., spray painted) will make a nice bike look like a jalopy of a ride.
You can also approximate this by putting a plastic grocery bag over the saddle, using ugly/old bar tape, fenders that are scratched up, and so on.

More here: Reduce Resale Value to Avoid Theft?

Quick-releases
Wheels and saddles with quick-release hardware are a target for thieves. Much of this can be combatted by locking them up properly (feed a cable through the saddle rails or the spokes of the wheels), but consider replacing these with hardware that needs a tool to open. Maintenance will take longer, but locking up (and unlocking) will be quicker.