LED circuit stopped working when expanded
When you have 31 LEDs off, your current draw from the 3.3V rail will be:

$$
I=31\frac{3.3V}{250\Omega}=409.2mA
$$

Whereas with only 3 LEDs off (that's what happened when you only had 4 channels):

$$
I=3\frac{3.3V}{250\Omega}=39.6mA
$$

Because you are pulling up the N-channel MOSFETs with low valued resistors, having to turn off 28 additional LEDs adds 360mA to your current requirements. If your power supply can't do that, it will start pulling down the voltage, and then the MAX6964 won't be able to output 3.3V to drive the gates (because it won't be getting them at its Vcc pin in first place).

This adds up to the fact that Si969 is only marginally capable of switching at 3.3V. The RdsON at VGS=4.5V is 3 Ohm. At 3.3V you can expect it to be higher, but it may still let enough current to light up the LED. However, if the power supply gets pulled down to, let's say 2.5 V, RdsON may be too high (higher that the 10 Ohm resistor of each LED channel, possibly) or it may not be able to switch at all.

What can you do about it?


Redesign your circuit using P-channel MOSFETs and active HIGH outputs from MAX6964. This will reduce the consumptions from your pull-ups no almost zero (only the pull-ups from the channels turned on will draw current). This may be traumatic for you so you can do another thing instead...
Increase the value of your pull-up resistors. 10x, even 100x. 250 Ohm is way too low. Unless you need blazing fast switching, a 4K7, 10K or even 22K pull-up is perfectly fine.
As a last resource, increase the current rating of the power supply (=buy a more capable one). Not recommended!