Definition of Pareto efficiency and prisoner's dilemma
(A, B) and (B, A) are in fact Pareto efficient.
I believe that your confusion may be because when discussing the Pareto inefficiency of the Prisoner's dilemma equilibrium, we always discuss (B,B) as the Pareto efficient alternative to (A,A) and (almost) never discuss (A,B) or (B,A).
Keep in mind that Pareto efficiency always requires a starting point to be evaluated. I.e. one way to determine if a case is Pareto efficient is to ask &quot;Starting from this point, is there any other allocation that is a Pareto improvement?&quot;. Here a Pareto improvement is a case making at least one person better off and no one worse off.
A natural starting point of discussion is the Nash equilibrium (A,A). Starting from there, only (B,B) is a Pareto improvement, which suffices to show that (A,A) is not Pareto efficient. That is what economists like to emphasize about the Prisoner's Dilemma and why textbook discussions focus on (B,B).
However, if your starting point is (A,B) or (B,A) then there are no Pareto improvements possible, as you argue. Hence, both those cases are Pareto efficient as well. However, note that these are not very interesting starting points anyway, because they are not Nash equilibria and therefore a bit irrelevant to discuss.