Is there a name for subduction boundaries at the North of Australia?
I do not think there is one name for the area. Rather it is a region with multiple plates, namely the Timor Plate, the eastern portion of the Banda Sea Plate, the Maoke Plate and the Woodlark Plate, all of which lie on the Arafura shelf which lies upon the Australian Plate. The Timor trough, Tanimbar Trough and Aru Trough would cover the western portion of the area you are questioning. 
 
The Papuan Thrust and Fold Belt (PTFB) and the Aure Trough make up the northern and eastern portion of the area in question. Hope this helps.