How to detect if video had been sized up (Scaled up) in resolution?
May i rephrase your question as

How natural does an image look?

If so, you are finding Naturalness Image Quality Evaluator (NIQE). Plz take a look of Naturalness Image Quality Evaluator (NIQE) no-reference image quality score. Then you may find some information from Laboratory for Image &amp; Video Engineering and then you will get some tools like

scikit-video
Video Quality Indicators

Good Luck!