Reinforcement learning with PPO: rewards decreasing
I had the same problem where the reward kept decreasing and started to search for answers in the forum.

I let the model trained while I search. As the model trained, the reward started to increase. You can see the tensorboard graph for rewards in validation time.

. 

The fall continued until around 100k~ steps and did not change a lot for 250k~ steps. 
After 350k~ th step, it slowly started to increase. Without knowing your number of steps trained, I would suggest training for more steps.

Also, I read about this (Reward first decreasing and then increasing) in an RL paper, if I find it I will mention it here.