Can we configure local and azure servers in a WSFC for Always On Availability Group?
Yes, you can
I've had 2 local (on-premise) SQL Servers and 1 VM in Azure, configured in Availability Group of three servers