Is a scholar whose work contributes substantially to my proposed project a suitable arm's length referee?
From what you describe there doesn't seem to be a problem, but who can say how others would evaluate the situation. But, consider:
You have no personal relationship to this person. They haven't contributed directly to your work other than through publishing their own work.
In fact, I would consider such a person to be an excellent judge with the necessary skepticism in making any evaluation. They know their own work (hopefully) and are in a good position to evaluate extensions of it.
But, it might be useful if that person also attests to their independence in any communication with the sponsor.