Defining an average in time for a density in time
I think it is implied that $\int_{-\infty}^{\infty}|s(t)|^2 dt = 1$ (which is why the author refers to $|s(t)|^2$ as a density).

I agree with your sentiment "I've always thought about an average as dividing by some base", but in this case, the base would be 1, so no division is needed. If you were to plot $|s(t)|^2$ as a function of $t$, and think of this plot as a histogram, the the integral:

$$\int_{-\infty}^{\infty}t|s(t)|^2 dt $$
will tell you the average value, since each value of $t$ is weighted by the density function.