Creating encrypted backups inhouse to be restored at client
Created the certificate from the .cer-file […]


You created the certificate but you didn't include the import of the private key, so it makes the use of this certificate to decrypt the data impossible.


  why I have to transfer the certificates private key to my client.


In this case, the certificate (which at its heart is just an asymmetric key) uses the public key to encrypt the data and the private key to decrypt the data. See my other response above.


  Doesn't that defeat the whole purpose of encryption?


No, but it defeats the purpose of backup encryption through SQL Server for your use case. Since you want yourself (1st party) and some random 3rd party to be able to encrypt and decrypt the data, you either need to agree to a set of keys to be used (which would put your other internal items at risk) or use a different method outside of SQL Server backup encryption where you can specify other encryptor options.

The fact that they can't decrypt the data without the required information is the whole point of encryption. In this case, it's doing the job properly it's just your requirements aren't meant to use it in this manner.