Why does Clipped Surrogate Objective works in Proximal Policy Optimization
Ok, so I think I have a better understanding of this now.

Firstly, let's remind the main idea of the PPO : staying close to the previous policy. It's the same idea than in TRPO, but the L function is improved.

So, you wanna make "small but safe steps". With clipped surrogate objective, you don't give too much importance to promising actions. You learn that bad actions are bad, so you decrease their probability according to "how bad" they are. But for good actions, you only learn that they are "a little bit good", and their probability will be just slightly increased.

This mechanism allows you to perform small but relevant updates of your policy.

hope this will help someone :)