Why does this app not restore from Google Backup until I specify the set in adb?
bmgr restore package doesn't reinstall an app: it sends a signal to the installed app to make it restore its data from backup. If the app isn't installed, you get the error message you saw.

It's the Play Store app that's responsible for installing other apps. If you like, you can think of the list of installed apps as Play Store's backup data, and it restores from that backup by installing all the apps in the list. This takes some time and happens silently in the background, since the backup mechanism is designed to be invisible to the user.

It's possible that Play Store managed to install your target app in between your first and last bmgr restore commands, or that toggling bmgr enable triggered it to reinstall the app at that point.