Does listener gets affected when we replace secondary replica with another server in Availability Group?
The listener is a cluster ressource. If you new server is part of the cluster and is added to your availability group, it should work just fine.