Multiple AGs on same Windows Failover cluster the Listeners don't work properly
It sounds like you have set the listeners to use the same port, probably the default 1433.

When you create an AG listener you specify a port for it to listen on. If you have multiple listeners on multiple AGs, they will each need to listen on a different port otherwise you will only be able to connect to the listener that started first and gained control of the port.

You should also ensure configure your SQL Server instances to listen on a specific port, as named instances will use a dynamic port by default. Specifying the port helps you document and control your environment better.

Info on configuring SQL Server to use a specific port

For your listener, run the below query against the primary of each AG to get the current port configuration:

SELECT ag.name AS [Availability Group], agl.dns_name AS [Listener DNS Name], 
agl.port AS [Port]
FROM sys.availability_group_listeners agl 
INNER JOIN sys.availability_groups ag ON agl.group_id = ag.group_id 
ORDER BY ag.name, agl.dns_name 


And use this code to update your listeners to use a specific port

USE [master] 
GO 
ALTER AVAILABILITY GROUP [** AG name here **] 
MODIFY LISTENER N'** Listener name here **' (PORT=1433); -- or change to whatever port you are wanting to change to. 
GO 


Info on changing listener ports here