Is it OK to use SQL Server availability groups on VMware installations? Can I use SQL Server availability groups on VMware installations or should I use a hardware installation for the Windows Server Cluster?
Sure, you can use Always On Availability Groups inside VMware virtual machines (or Hyper-V, or Xen, or cloud provider VMs, etc.)

They're actually a great fit for high availability and disaster recovery for virtual machines because each VM has its own copy of the database.

Keep in mind that if you're trying to avoid single points of failure, then don't put all replicas on the same shared storage, along with your backups. As the great philosophers once said, you gotta eep 'em separated.