Automatic switching to the changed set of TTL inputs
I think you have found the simplest way to do it in discrete logic.
You could perhaps use a 4 input XOR to combine your inputs and then use a single flank detector, but this could fail if two buttons are activated at exactly the same time.
You could perhaps use a 5 channel mux and find a way to use the spare channel as the flip-flop
As you only need your logic to be faster than a finger it may be easier to use a cheap mirocontroller module like an arduino uno clone.