Is there multi-agent reinforcement learning model in which (some of the) reward is given by other agent and not by the external environment?
This is mostly an implementation architecture problem, and the thing is that basically you can implement anything in the traditional setting. To do so instead of having Env&lt;-&gt;Agent1&lt;-&gt;Agent2, you should have Agent1&lt;-&gt;SuperEnv&lt;-&gt;Agent2 where SuperEnv contains Env, and simply uses the reward given to SuperEnv by Agent1 and passes it to Agent2.
I know this might seem a little counter-intuitive when comparing the implementation to the real-world problem setting, but the consistency of the RL structure (i.e. Environment that interacts with all the agent) is very important for your solutions to be easily understandable by others.