Is it possible for an applicant to be disadvantaged by negative comments sent via emails without his/her consent?
This is a supper difficult problem and is responsible for much of what is wrong with academia. 

Its definitely not unheard of for a recruiting academic to call contacts at institutions a candidate has been associated with and ask. 

If you are in Europe, and you are turned down for a job you were sure you were the best candidate for, you can issue a GDPR demand to the universities in question, and they must turn over any materials that contain information about you. This should include both formal and informal references. Obviously if the bad-mouthing happened over the phone, there will be no record of it, which is exactly why people often use the phone for this sort of thing. 

You could then try claiming that the university treated you unfairly in the hiring process. You'd have to be careful because the university could try saying "We didn't select this candidate for other reasons, nothing to do with the reference. Here is the documentation that lists the criteria we used and the candidate's scores". 

You could also try suing the phantom referee for libel/defamation if you were really into long drawn out legal battles.