Difference between equivalent output current & secondary RMS output current for flyback
As you can see, if using 1st equation, we can get output power 50W,
  while if using 2nd equation, we can get output power 96W.


No, you can't use the 2nd equation meaningfully to obtain output power. 

The secondary RMS value of 3.97 amps is the current flowing through the secondary winding and, it may have an average (DC) value that is somewhat related to your DC output current of 2.08 amps but, as an RMS value, it is meaningless to what DC current the final output stage can deliver.

That secondary winding current is a partial triangle/saw-tooth wave raised to a DC value. The clue to it being a partial triangle/saw wave (a basic standard for flyback converters) is the division by 3 in the formula and the square root in front of that part of the formula.

Here's a pictorial example of the input and output winding current waveforms for a 2:1 step down flyback circuit operating in DCM. I think your circuit is probably more like 10:1 and CCM but, the general shape of the waveforms are the same: -