What are proxy reward functions?
In the paper that you cite, Inverse Reward Design (2017), the authors actually define what they mean by &quot;proxy reward function&quot;.

We formalize this in a probabilistic model that relates the proxy (designed) reward to the true reward

So, the proxy reward function is the reward function designed by the human, which may not necessarily be the reward function that he/she intended (i.e. it may be a misspecified reward function), given that the human may have forgotten to model/incorporate certain (unpredicted by the human) scenarios or situations that the agent may face. This usage of the word &quot;proxy&quot; is thus consistent with the general usage of the word in computer science, i.e. a &quot;proxy reward function&quot; is a reward function that is used instead of the intended (optimal) reward function.