Avoid user login Directly to AlwaysOn High Availability Primary
There are a few questions that are similar to what you are asking, may be board-line duplicate but either way:


Protect SQL Server from hackers
Limit connection rights to SQL Server


That second question listed is probably the duplicate of this one.

Overall if you are looking at ensuring logins can only access via the listener you are going to have problems trying to use a login trigger. I am not sure you can pick up DNS or IP address it used to actually connect to SQL Server. If it can then it would be a viable option.

I would opt to use your firewall or network infrastructure. A firewall can be much more efficient and easier to manage with blocking access to either the IP or access from an IP range to a specific IP.