Is it possible to have audio follow window across multiple displays
This is not possible.

Apple does not "link" audio devices and video devices in any way.  As far as macOS is concerned, you have discrete audio devices and discrete video (display) devices.  In fact, you cannot even control audio from a Mac connected via HDMI or DisplayPort

Some Apps (i.e. Vox Music Player) allows you to select a different audio device from the default.  If the App you're using (Safari for example) is playing a YouTube video, it would have to be able to be "aware" of the other audio devices (it's not).

Your app would not only have to be aware of the additional audio devices but the location of it's Window in relation to the monitor it's connected to.  It's highly unlikely app programmers would include that functionality.