Setup Time, Hold Time - What is the underlying principle for having them?
You're forgetting a couple of important facts:

A flip-flop isn't a single atomic gate, but made up of multiple gates.
It takes time for a signal to pass through a gate (or propagate).
There is no such thing as a pure square wave.

Take this diagram of a transparent latch:

Assume each gate requires one &quot;time unit&quot; to propagate the signal.
The D signal arrives at the input to one NAND gate at \$T_0\$. It also arrives at the NOT gate at \$T_0\$. It then leaves the NOT gate and arrives at the second (lower) NAND gate at \$T_1\$.  So the shortest time before the CLK signal can have any effect on the input NAND gates will be \$T_1\$ since that is when the D signal has propagated to them.  SO then the output of those NAND gates will be propagated as stable at \$T_1\$ for the upper one, and \$T_2\$ for the lower one.  Then it's on to the next two NAND gates. Those again add 1 time unit to each of the signals. On top of that, the outputs of those then feed back into each other's inputs, so as they change they propagate a new signal through themselves, each adding another time unit.
Only when the outputs have stopped sorting themselves out will the gate be &quot;stable&quot;, and that is when it's in the &quot;hold&quot; state.  It could be many time units.
Then of course you have the &quot;square&quot; wave, which is far from square.  Each change from low-to-high or high-to-low takes time. Only once a signal passes a certain threshold will it be seen as either high or low. Different factors affect how long those transitions take, including the gate capacitance of the MOSFETs in the logic gate, the actual switching time of the MOSFETs, etc.
So you can see there is a certain amount of time taken for a flip-flop to change from one state to another, and at different points during that time different things happen with the gate.