number of queries to recover the permutation
Suppose that my input is some permutation $\sigma$. Applying $f$ is the same as multiplying by $f$ from the left (assuming that $\alpha\beta(i) = \alpha(\beta(i))$, and then applying $p$ is the same as multiplying by $p$ from the right. In other words, we have an oracle for the function $\sigma \mapsto f\sigma p$.

Given $f\sigma p$ and $f\tau p$, we can find $(f\sigma p)^{-1} (f \tau p) = p^{-1} (\sigma^{-1} \tau) p$. In particular, using two oracle calls we can find $p^{-1} (1\;2\;3\;\cdots\;n) p = (p^{-1}(1)\;p^{-1}(2)\;p^{-1}(3)\;\cdots\;p^{-1}(n))$. Using one more oracle call, we can also find $(p^{-1}(2)\;p^{-1}(1)\;p^{-1}(3)\;\cdots\;p^{-1}(n))$ (the only difference is the first two values). From these two long cycles we can locate $p^{-1}(1)$ and so recover $p^{-1}$ (and so $p$), as well as $f$ if we so wish. In total, we used three oracle calls.