Unable to use RGB matrix and the serial monitor at once
According to Adafruit https://learn.adafruit.com/32x16-32x32-rgb-led-matrix/overview these led matrixes require active scanning. It means that the RGBMatrixPanel will set up some scanning interrupt to refresh the screen periodically.

This can very much interfere with serial receive, as according to Adafruit, the panels require a serious data transmission all the time.

Try the followings:


have one byte serial commands, e.g. N for "on" and F for "off" (I know you will need more complex commands, but this is just to verify the theory)
remove all debug Serial.printlns (those also consume some time)


So try this:

void loop() {
  if (Serial.available() &gt; 0) {
    char incomingByte = Serial.read();
    if (incomingByte=='N') {
       matrix.drawPixel(0, 0, matrix.Color333(7, 7, 7));
    }
    if (incomingByte=='F') {
       matrix.drawPixel(0, 0, matrix.Color333(0, 0, 0));
    }
  }
}