How to properly pull up a single-relay module
Your problem isn't the lack of a pull-up at the transistor collector.

The problem is R1. 

When the micro starts up, its IO pins will be in high-Z state (input mode) until the firmware gets around to changing them to output mode. In that time, R1 provides a current path from the regulator U1 to the base of the transistor Q1, which turns it "on", drawing current through the relay coil.

Adding a pull-up at the collector of Q1 will just increase the amount of current drawn when Q1 is on, increasing heat dissipation. The relay will still be energized during start up, because there will still be current flowing to the base of Q1.

To avoid this problem, remove R1. This should be enough to solve the problem.

If you want both belt and suspenders, add a pull-down (perhaps 47 kohms) from the GPIO2 pin on JP2 to ground.

Edit

In comments you said, 


  R1 is required for the ESP module to boot correctly, both GPIO 0 and 2 have to be HIGH for that matter.


If GPIO2 is high during start up, the relay will be energized during start up (because Q1 will be "on").

You may even have a problem with the logic level during start up because R3 and the Q1 base-emitter junction form a pull-down in competition with R1, so that the GPIO won't be seeing a very strong HIGH during start up.

I'd recommend either 


Find another IO pin on your micro that doesn't affect the boot sequence, 


or


Add an inverter, either a single transistor or a 1-gate logic chip, between the IO pin and Q1.