Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?
In RL we have:


Actor-only methods such as REINFORCE in which the output is a probability distributions over actions. REINFORCE is a policy gradient method but doesnt use a critic.
Critic-only methods such as Q-learning in which the output is the expected reward for every available action ($Q(s,a)$ $ \forall a\in A $)
Actor-Critic methods that involve both Actor and Critic estimations. For example the popular DDPG and A3C algorithms. Both algorithms are policy gradient methods. By reading the papers you will start getting a sense on why the simple REINFORCE introduces variance in gradient estimations and how a critic can reduce it.


Policy Gradient methods are based on the Policy Gradient theorem. A standard implementation is an Actor-Critic algorithm, and we use both Actor (probability distribution over actions) and Critic (Value functions) in order to trade-off bias and variance in your gradient estimations.