Policy Gradient custom loss function not working
There could be many reasons. Custom loss functions are difficult to get right.

One conceptual problem is that the policy of the agent should not be the label. The label should be the reward signal from the environment. If the label is the reward signal from the environment, the agent would learn which policy would maxim rewards. If the policy of the agent is the reward, the agent would "chase its own tail" and not learn environmental rewards