Generation of 'new log probabilities' in continuous action space PPO
The idea in PPO is that you want to reuse the batch many times to update the current policy. However, you cannot update mindlessly in a regular actor-critic fashion, because your policy might stray too far away from the optimal point.
This means you repeat your step 6. epoch amount of times for the same batch of trajectories. Usually epoch is somewhere between 3 and 30 but it is a hyper-parameter you need to adjust. For the first repeat, the old and the new policy are the same, so their ratio should be 1. After the first update, the new probabilities will change due to the updated policy, whereas you will still need to use the old probabilities giving you a ratio different from 1. The old probabilities will stay the same during these epoch update steps, whereas your new probabilities will keep changing.