A basic level question about grounds in a differential amplifier circuit
But where is the common ground for the differential input case?

It can be anywhere! Assume that the average voltage of the two input voltages is at common ground, so 0 Volt. Now what is the differential input voltage ? It is:

\$V_{in,diff} = (V_{inp} - 0) - (V_{inn} - 0) = V_{inp}-V_{inn}\$

Now I use a different common ground which is at + 12.3 V, what do we get now:

\$V_{in,diff} = (V_{inp} - 12.3) - (V_{inn} - 12.3) = V_{inp}-V_{inn}\$

See, no difference!

Since it is the difference between \$V_{inp}\$ and \$V_{inp}\$ what counts, whatever the common ground is, it is added and subtracted so the net result is zero.

The common ground is irrelevant. Also they do not need to be tied although in practice they often are. Ethernet (network) cables for example use differential mode signals. The ground does not need to be connected. On both sides of the Ethernet cables small isolation transformers are used to re-define the ground level so that it suits the local circuit.

Also in practice the voltage between inputs and the ground of the circuit will be limited for example by the input voltage range of the amplifier.