How to choose a correct LED PWM driver?
You could use a transistor driver for each input (LED):



Source: electronics-tutorials.ws

In your case the second circuit suits (low-side driver or current sink) and \$I_R = 100mA\$.