WindowServer High CPU using external monitor - Big Sur
If you're using an usb mouse, I've noticed that my gaming mouse causes the WindowServer to use a lot of CPU.
Turning transparency down in accessibility settings has helped quite a bit.
As of Dec 9th, 2020, it looks like it's still happening intermittently with one 4K display and my 2019 Macbook Pro.