Relay module and internal pull-up resistors
Connecting one side of a switch to ground and the other side to an input pin (with its pinmode set to INPUT_PULLUP) is a standard way of connecting control switches to an Arduino installation, and is not a problem.

Note, claims that built-in pull-ups are 20KΩ oversimplifies the situation.  See, for example, Table 28.2, DC Characteristics, in the ATmega48PA/88PA/168PA/328P specs, which shows a min value of 20KΩ for   I/O Pin Pull-up Resistors and a max value of 50KΩ.  I've measured resistances of about 40KΩ, IIRC, whenever I've measured the pull-up values.

Note, built-in pull-ups typically are of too much resistance to work as pull-ups for TWI/SPI/I2C communications.

Note, when you say in the question that you connected the relay-modules to input pins and they worked ok, I suppose that could be the case, but in general it is better to put i/o pins into output mode when you want to drive an output.  When you send LOW to an input pin, its built-in pull-up is turned off, the input impedance becomes high, and then the pin can float to any value between ground and Vcc. It might fail to turn off an attached device if that device also has high impedance.