How to evaluate the research level of a paper before any publication?
While I think your "justice" metaphor is wrong, the answer in all cases is experience. You need experience in the field as a reviewer. You need to know what has been done and what is important yet to do. As an editor you need either that field experience or experience with your reviewers - who is trustworthy and who is not. As an author in the early career you don't have any experience other than some within the field, but it will grow. 

For everyone here, the way you get experience is to make best effort attempts at whatever job you have and evaluate the response. As they say in engineering and computer science, "Good design comes from experience. Experience comes from bad design." 

If you are looking for an algorithm, then I would say it doesn't exist. It might be possible in theory to construct one with an AI looking at tens of thousands of interactions, but it might show bias, as many such things have been shown to do. 

But the system as a whole just depends on (nearly) everyone trying to do their best with what they have in front of them in world of imperfect information. 



In theory, theory is the same as practice. But not in practice. - Fnord Bjørnberger