Windows Server 2019 non-owner node cannot see Cluster Name or IP
I managed to fix this by enabling HA ports in the Azure Load Balancer (standard Load Balancer). That seems to have fixed the issue.
By the way I have SQL Server 2019 Enterprise on Windows Server 2019.
Set the following Load Balancing rule in the Azure Load Balancer for WFCS.

The source of this information for this article : https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually-configure-tutorial#add-the-cluster-core-ip-address-for-the-windows-server-failover-cluster-wsfc
The article does not talk about HA ports because it is a Standard Azure Load Balancer feature and the article was written using a Basic  Azure Load Balancer.