Does policy optimization learn policies to make better actions with higher probability?
The image in your question looks to me like a loose hierarchy explaining how various Reinforcement Learning methods relate to each other. At the top are broad categories of algorithm based on whether they are value-based or policy-based, towards the bottom are more specific methods.

There is more than one way to categorise and split RL algorithms, and it would be messy to try and include all the ways that they relate to each other. Just bear in mind that it is a very rough guide.

The main difference between value-based and policy-based methods is:


Value-based methods learn a value function (by interacting with the environment or a model of it), and consider the optimal policy to be one that takes actions with maximum value.
Policy-based methods learn a policy function directly, and many can do so without considering the estimated value of a state or action (although they may still need to calculate individual values or returns within episodes).


Note Actor-Critic is linked to both headings, as it learns both a policy function (the "actor") and a value function (the "critic").


  I would like to know is it correct to say: Policy Optimization learns policies to make better actions with higher probability?


Yes that is broadly correct, although you don't define "better". A policy function typically returns some probability distribution over possible actions for any given state. When learning, it will tend to increase probabilities of actions that resulted in better returns (discounted sums of rewards) and reduce probabilities of actions that did not. This may be a very random, high variance process though, depending on the environment.

There are exceptions. Some policy-based methods learn a deterministic policy $a=\pi(s, \theta)$, and adjust the value based on adding some noise to this function to explore and adjusting the function based on the results of the different action. These don't behave like your statement (because there is no probability to make higher).


  Also, what is the location of Proximal Policy Optimization in the picture?


Proximal Policy Optimization is definitely a policy-based method, and it does use an estimate of a value function too for updates (in this case a value called advantage which you also see used in Advantage Actor-Critic). 

In the diagram I would probably place it under Actor-Critic Methods in a new row as a specific example of one. However, it does have some significant variations from "vanilla" Actor-Critic, based on how it restricts large changes to the policy function.