Availability Group Listener I'm looking at AlwaysOn availability groups. The more I look at it the more it appears that the availability listener group is a single point of failure. Where exactly does the listener actually run? A separate a server, the primary SQL server, all of them?

Say I have a whole application stack at my second data center. How do I configure the listener(s) such that they will be running at both sites and that the applications will point to their own local copies? 

I'm sure I'm missing something here but I don't know what.
The AG listener is a virtual network name (VNN) or virtual IP (VIP).  


  An availability group listener is a virtual network name (VNN) to
  which clients can connect in order to access a database in a primary
  or secondary replica of an AlwaysOn availability group.


You point applications to the listener (which is registered with DNS) and directs traffic in the AG.  So, unless all your DCs go down at the same time, your redundancy is the number of DCs/DNS servers.


  An availability group listener consists of a Domain Name System (DNS)
  listener name, listener port designation, and one or more IP
  addresses. Only the TCP protocol is supported by availability group
  listener. The DNS name of the listener must also be unique in the
  domain and in NetBIOS. When you create a new availability group
  listener it becomes a resource in a cluster with an associated virtual
  network name (VNN), virtual IP (VIP), and availability group
  dependency. A client uses DNS to resolve the VNN into multiple IP
  addresses and then tries to connect to each address, until a
  connection request succeeds or until the connection requests time out.


This MSDN article (which 
i quoted from) should be a good starting point for what, exactly, an AG Listener is.

https://msdn.microsoft.com/en-us/library/hh213417.aspx