Can I break batteries by putting them in the wrong way?
Consumer Devices

Almost all consumer electronics have protection in place to prevent any damage when connecting the battery backwards. Companies know they can't trust consumer to put the batteries in the correct orientation and it is usually cheaper for them to add the protection then it is to deal with support calls and returns.

Batteries in your own device

If you are wanting to make your own device, you will need to add protection or it will be almost guaranteed that you will have ICs that smoke. The battery itself probably wont be hurt. They can output ALOT of current. All of your ICs and polarized caps will usually break in such away that they will act like an open circuit. Because of this the batter will only have a short time where it is having to put out a lot of current.

How to protect your device

The easiest way to protect your electronics is to put a fuse in-line with your battery. This is a good all-purpose fail safe. Fuses are too slow to blow to protect all ICs, but at least it prevents your project from catching on fire.

If you place a single diode in series with your battery you can prevent any current flowing in the wrong direction. Do be prepared to see a 0.7v drop across the diode.

You can also use a 4 diode configuration to allow your circuit to continue to work regardless of how to battery gets connected. This disadvantage of this method is that you have 2 diode drops (about 1.4v).