Which reward function works for recommendation systems using knowledge graphs?
I found the answer further into the paper! I'll post it here for everyone.

Given any user, there is no pre-known targeted item in the KGRE-Rec (Knowledge Graph Reasoning for Explainable Recommendation) problem, so it is unfeasible to consider binary rewards indicating whether the agent has reached a target or not.

Instead, the agent is encouraged to explore as many “good” paths
as possible. In the context of recommendations, a “good” path is one that leads to an item that a user will interact with, with high probability. To this end, a soft reward is given only for the terminal state $s_T = (u,e_T,h_T)$ based on a scoring function $f(u,i)$. The terminal reward $R_T$ is given as - 


where the value of $R_T$ is normalized to the range $[0,1]$. Details about $f(u,i)$, the scoring function, can be found in the paper.

References:


Reinforcement Knowledge Graph Reasoning for Explainable
Recommendation [Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard
de Melo, Yongfeng Zhang]