We started by saying only math and code have verifiers. Tulu showed that we can use something like Ifeval to verify stuff like output format etc. Then, why cant we just use llm as a judge as an “external verifier” for everything?

for eg essay writing. Get the model to first cot an essay plan, grade this plan, grade individual paragraphs/points.

Isnt this basically the same as RLAIF? If it is, why can’t we train reasoning models for open domains too in the same manner? Is the problem with this basically that models are bad at llm as a judge?