Sort a View by whether content has a user reference
I have an idea!  Use your current block to limit the list to users who are referenced in the node, and then add an ATTACHMENT to the block listing users who have flagged it who weren't referenced in the node.

That way you can separate them our and sort independently, but they'll still appear as if they're displayed in the same view.

Here are the exact steps:


Add an additional relationship that is the node user reference field, but reversed: 





I added the 'Content: nid' field with the above relationship to my view just while testing, and that was helpful, so I'm including it as a step...
Add that field 'Content: nid' using the user-reference relationship to your set of contextual filters too, if you're using it, to make sure the content they're referenced in is the current node (don't want to sort them first just because they're referenced in SOME node on the site, right? ;-))





Add an attachment to your view, and set it to display directly after your existing page or block.  In this one, you'd want to do the exact same as above, however, when you're setting the contextual filter, select to 'Exclude' the matching answers (you'll find that check box in the collapsed 'More' section at the bottom.