Uneven wear of brake pads
With the brake tension released (either through the quick release or untensioning the brake wire) is the spring tension on both sides equal? As @Daniel R Hicks notes, some unevenness is inevitable with direct pull and cantilever brakes due to their design. 

For V brakes and other direct pull (as well as cantilever) there's a little screw on each arm that helps adjust the spring tension. For cantilever brakes, you might also want to make sure your straddle cable is evenly centered. 

Also, you should make sure both brake arms move smoothly by lubricating the pivots with a dry lubricant (PTFE, etc.) and working them back and forth.  There's a related question about how to overhaul the sleeve bearings: cleaning v-brake arms with riveted inner sleeve