Raspberry Pi Relay only working tapping it
The board you are using is intended for 12V VCC, and has 12V relays. A typical relay may only trigger on 80% of its nominal coil voltage, but it will hold at 40% of the coil voltage. This means that when you power it at 5V, it's not enough to turn on, but once you physically force the relay to close, then the 5V is fine.

While there are ways to use the existing board, your best option is getting a 5V relay board.