Finish current album and queue next album
You would need to create a playlist with those two albums, then when you play it, it will loop the whole playlist.

(I tested using the Up Next feature but it will not loop tracks played through Up Next.)