Options for adding “C” wire to thermostat
Once you confirm the power supply in the furnace matches your thermostat's requirements, then yes, use the extra wire. Not only confirm a constant 24v is supplied, but whether it should be AC or DC. (DC would be really weird, but worth confirming)

Since the wire you use may not match the usual color coding, label both ends of the wire to eliminate any confusion in the future.