What differences in effect does fast repetitive exercise have compared to slower muscle-building exercise?
Your body creates muscle after an exercise through a process where it fuses muscle fibers together to form new muscle protein strands called myofibrils. these myofibrils increase in size as workout intensity increases, leading to muscle growth. Muscle growth occurs when muscle protein synthesis happens faster than muscle tissue breakdown 1. 

Since you are never really increasing the amount of strain on your muscles(i.e. by adding more weight to a workout) there is no incentive for your body to create more myofibrils, rather it will just replace the ones that were torn during your drumming.