charge a 15v Cap with 24v
Yes, the concept could work to disconnect the 24V charging supply as long as you had arranged the charging time constant to give sufficient time to detect the voltage over 15V and get the supply disconnected. 

What this means is that you would not want to be trying to charge the capacitor with it directly connected to the supply. In that case it would charge so fast that some detector comparator circuit, which might have a 1usec response time and a switch MOSFET which may also have a response time, may not have been able to register the 15V event before the cap was charged farther than 15V.

So use an series R to charge through so that you limit the charge rate giving time for the detect circuit.

You have another choice to make a voltage divider of two resistors that divide 24V down to 15V. Then connect your capacitor across the lower resistor of the divider and it will only charge to 15V.