Command runs fine in elevated command prompt, but fails when run in a Windows Task Scheduler task
Thanks guys I resolved the problem. Although I'm not sure what I did that resolved it. What I did:

Created a brand new SQL Server login with sysadmin rights (I know it shouldn't need that much, but hey, I needed to fix this.)
Added a new local user login in users and computers MMC in the SQL Server hosting the machine, and made it a local admin to the machine.
Updated everything to use those users, instead of what we had before, and suddenly it started working.
So, it's something related to permissions, most likely for the SQL Server login. But still, as far as I can tell the initial user had everything it needed. So much so that I could confirm that to be the case both using a command line as well as a batch file.

Thanks for trying to help. Closing this here. Best, Raphael.