Startup voltage help: How can I change the input voltage to this IC when starting the circuit?
I am attempting to start u12 on the input voltage, then when the
regulator starts up, to use that to turn off Q1, changing the input
from the 24v line to the 15v line. However, I don't think that would
work

You are correct, that won't work. The FET will be turned off when the 15v rail is low and turned on when it is up - the opposite of what you want. Also since it is a junction FET a significant amount of current will flow from the Gate to the Source.
But you don't need to switch off the 24 V supply. During startup the UC3842B draws less than 1 mA, so a simple resistor is all you need to get it going (eg. 6.8 kΩ would drop the supply voltage to 17.2 V at 1 mA). The higher current draw when running will automatically pull the voltage down until the 15 V supply takes over through diode S1. A small portion of the operating current will still be supplied by the resistor (~1.4 mA with 6.8 kΩ), but this doesn't heat up the IC any more than running it directly from the 15 V rail.

The only other change needed is an input filter capacitor (eg. 47 uF) from Vcc to ground. This stabilizes Vcc and slows its rise time to (hopefully) give the 15 V supply enough time to get up before the UC3842 starts up.