Is it wrong to assume the op-amp has the same voltage at its both terminals when deriving its non-inverting closed-loop gain?
Yes. V+ = V- in cases whenever an OP-AMP gives out stable output and negative feedback dominated cases. In the real world, they are not equal. But almost equal. The difference between V+ and V- is almost zero and the gain of the OP-AMP is huge. But for a theoretical purpose, we just take it as zero and assume the gain of the OP-AMP is infinity.

If you want a better answer, use the formula V0 = A.(V+ - V-). In the video, V+ = Vin and V- = R1.V0/(R1 + RF) and then derive V0.