What is the Meaning of the Notation
It means that there exists a polynomial $f(x,y,z)$ such that the algorithm runs in time $f(|S|,n,\frac{1}{\epsilon})$.

Specifically, it means that there are constants $c_1,c_2,c_3\ge 0$ such that the algorithm runs in time $O(|S|^{c_1}\cdot n^{c_2}\cdot (\frac{1}{\epsilon})^{c_3})$.