What is the difference between 'Set To Frame Size' and 'Scale To Frame Size'?
Scale to Frame Size

Scales the clip to fit the size used in the current sequence whilst retaining the original aspect ratio to fit within the frame. Essentially, the clip has been rasterized to match the resolution used on the current sequence. 

Despite adjustments to the size of the clip, the scale setting in the Video Effects panel for the clip will remain as 100%. This isn't a permanent change, the modification can be disabled by unchecking the option in the same way you enabled it.

If I place a photo which has a resolution of 3264 x 2448 and used 'Scale to Frame Size' to fit it into a widescreen 1920 x 1080 resolution, the photograph will be reduced in resolution. If I wanted to zoom in by scaling the photograph back to 100%, I would lose the original quality.

'Scale to Frame Size' can improve performance when scaling down a large clip.

Set to Frame Size

Scales the clip to fit the size used in the current sequence.

Premiere will automatically apply the scale adjustment under the clips Video Effects, which can be adjusted. The clip retains the original resolution.





Here are two identical photographs, the first photo I used 'Scale to Frame Size' and scaled in the Effects panel by 400%, for the second photo I used 'Set to Frame Size' and scaled by the same amount. 

You should be able to see a clear difference in quality when scaled.





You probably won't see any noticeable difference scaling UP when using either of the methods.