Is this a valid matrix circuit, I've drawn it based on guides and examples online
This is not an answer to the question.

You will need to separate the switch and encoder grounds if you want to multiplex the encoder module switches.

You can add a second ground pin this way.