Flag : access, context, role
I have the impression that what you're asking for requires more then just (only) the Flag module. I.e. you should get it to work if you also use the Rules Link module.

If you're not familiar with Rules Link, then have a look at "How to show authorized users a link for relevant Views results using the Rules Link module?". The scenario explained in that question is about users that are "blocked" or "active", and which you could consider as a variation of what this question here is about, i.e.:


think of "active" users as "users who have flagged their nodes" (as in your question).
think of "blocked" users as "unflag a user from his view of applicants" (as in your question).


Of course, you don't want to block or activate users for your case. But instead you could use 2 separate flags:


a 1st flag to be used by all the users, to indicate something like "I want to apply for this node" (adapt the label to whatever fits).
a 2nd flag to be used by the recruiter to indicate something like "reject a user as an applicant".


Obviously, you could show to "recruiters" (role) a view of all the users who have flagged their nodes (with the "1st flag"). And using Rules Link you could improve that view to trigger some additional Rules magic to simply flag an applicant (with the "2nd flag").

Note: no custom coding needed, only 2 flags to be configured, and some rules magic (combined with Rules Link).