Can a 12v dry relay be able to handle 24v load with its own PSU?
It should be okay, but I would strongly suggest looking up the part number on the relay itself and double checking with the relay's datasheet. The link provided claims "FORM-C relay output-SPDT 5A@12V DC".

Generally, the coil voltage and the switching voltage are spec'd differently. A Form-C relay with a 12VDC coil could have a switching voltage rated for 250VAC/30VDC @ 5A. The exact rating for the switching voltage would be found in the relay's datasheet.