Carbon Brush Specific Resistivity and Wear
For the motor to be doing anything useful, there's current passing through the windings on the armature, and the windings are arranged in the slots, and in their connections to the commutator such that the current in (most of) the windings is producing positive torque. As the armature rotates, coils that are leaving the field under one pole and entering the field of the next one need to reverse current to maintain that direction of torque.

The windings each have some inductance, so some effort is needed to cause that current to collapse and then build in the opposite direction. Most of that is done with shaping the field, by shaping the tips of the pole pieces, rotating the brush gear some degrees, or on larger machines using additional poles in the field assembly. 

None of that is sufficient under all conditions, and there's always some energy left in the field around the winding in the slot, so as the commutator bar comes into contact with the brush, there's a potential there relative to the next bar that has the other end of the winding already under a brush (may be the same one, or a different brush of the same polarity depending on the winding pattern). The resistance of the brush acts as a sink for the energy so that the current can decay, so a larger resistivity allows the decay to be quicker, such that the current has decayed and reversed by the time the bar leaves the other side of the brush.

As you've found, the resistivity of the brush causes a voltage drop in the armature circuit, so the voltage at the brush face is lower, reducing speed and efficiency. There are brush materials that have anisotropic resistivity, produced by the pressing process, so that the resistance along the brush from the shunt at the back is low, but the resistance across the face is higher. Other brushes can have different materials sandwiched together to put the high resistivity material only at the edges. Very large machines may have multiple brushes slightly staggered around the commutator to achieve the same effect.