Why is my Soft Actor-Critic's policy and value function losses not converging?
I would say it is the nature of data. Generally speaking, you are trying to predict a random sequence, especially if you use the history data as an input and try to get the future value as an output.