Can I run a 12 volt automobile winch on a 20 volt power supply and how would I caculate if possible
Even though you failed to specify the load rating on a say 1~2 ton winch, you overlooked the basics of Ohm’s Law for motors with equivalent load resistance and power for no load + load.  I doubt you measured the DCR of the coil to see if that did not exceed the OCP limit of the supply. 

Consider ;
Is motor DCR &lt; V²/Pd=20²/90=4.4 ohms? Then the supply will fault out and shutdown.

The rule of thumb is the voltage source should be capable of supplying  Vbat/DCR without blowing a fuse and >10% of Vbat/DCR at rated load and max current about 1%  V drop desired and 10% V drop tolerated.

These simple equations and winch milliohm DCR test measurement will tell you must have at least a motorcycle battery and probably a car battery at 12V+- 20% and never 20V @4.5A. 

not even close

Edit ——
Plan B



Get a 500W+. PC ATX supply and use heavy gauge wire to the 12V supply 20A to 30A+ capacity. 5V may need a 5% dummy load of its rating to prevent under voltage trip on 12V. Gnd jumper to PWR enable on ATX connector is mandatory.