Does changing the electrolyte concentration in the electrolysis of water affect the speed that hydrogen gas is produced at the cathode?
With decreasing electrolyte concentration, either the needed voltage raises at constant current, either the passing current decreases at the constant voltage.
Both happens because at lower concentration, a higher potential gradient across the electrolyte is needed to maintain the same current, as the smaller ion count requires  ions to move faster for the same current..
This potential gradient and the resulting potential drop acts against the external voltage. It pushes forced electrode potentials closer to their equilibrium potential. The net electrode reactions are then slower.