Disable mouse interference in input source overlay when switching?
As I've never seen a keyboard with that 'globe' on the Fn key to switch layouts, I'm curious as to what your Control panel for it looks like.
System Prefs &gt; Keyboard &gt; Shortcuts &gt; Input Sources.
If that's anything like mine, just switch it to another key command. I use Ctrl/Spacebar &amp; like you I only have 2 languages, so it's a simple flip between the two.

This produces no screen overlay, but if you have the language menu bar item, you see the flags switch.