Generated CSV cannot be read by Feeds
It drove me crazy but in the end, I found the issue.

The website has a default language other than English. When I look at the CSV url (as a logged in user) I see the correct headers in the CSV (because my language is English). But when an anonymous user looks at the url, he sees the translated headers :) So the headers dont match with the ones I set up in the Feeds settings.