Whitespace created a start of HTML
I finally found what the problem was.

The translations files (/craft/translations/de.php) were encoded as "UTF-8 with BOM"
I opened these files in Sublime text and saved them as regular old UTF-8, which fixed the issue.