Linked Server already exists in sys.server
Issue was resolved.

Actually it was for some ETL process for which I need to access a server which is not in AOAG and a server which is part of AOAG.I have added the listener only in one remote server as Linked server but not AOAG servers.


  I have to add Listener as Linked server in each AOAG servers(Replicas) and the remote server which is a stand-alone
  to make the ETL process work.


Hope this helps someone.