What's this "Finder" icon on my Desktop and how do I get rid of it?
Relaunch the Finder app to get rid of the tiny window.

Go to Apple Menu → Force Quit... or press Option + Command + Esc. In the Force Quit Applications window that appears, click on Finder and then click on Relaunch button.

It will get rid of the tiny window.