Dice with an average roll of zero?
Probably the most commonly available ones are called Fudge dice, as used in the Fudge/FATE roleplaying games.

A Fudge die is a six-sided die with the following sides: minus, minus, blank, blank, plus, plus.

Games generally have players rolling a couple of Fudge dice together and adding them up, which creates a curved probability distribution centered around zero. 3dF, for example, gives you a range from -3 to +3, with 0 as the mean, median, and mode.

This Wikipedia article has more information.