What's the difference between polylogarithmic and logarithmic? I can't imagine one that is not polylogarithmic
but logarithmic.

$O(\log N)$ satisfies both.

What about $O(\log^{3}N)$, $O(\log^{100}N)$, and  $O(\log^{10000}N)$ ?

Let's say $N=10^{10}$
Definitions:

An algorithm is said to run in


logarithmic time if $T(n) = O(log(n))$
polylogarithmic time if $T(n) = O(log(n)^k)$ (also written as $T(n) = O(log^k(n))$)


That means they are the same for $k=1$. Otherwise they are different and your other examples are all polylogarithmic.
I'm not sure how exactly to explain what the difference is but maybe a picture will help you: