SSIS Merge Join: Delete does not work
The answer is actually pretty easy: 

You have to use a Full Outer Join instead of a Left Outer Join in the Merge Join Task. 

The referenced tutorials actually told me to do so, I guess I mixed it up with another tutorial that did not include the merge delete.