vorbis: fallback to normal division instead of crashing

The use of ff_inverse speeds up slightly arches + compilers that
do not provide a division faster than the whole machinery, such
as ppc32 + gcc4.7, but has operational limits.

Drop the always-enable assert and provide a fallback.
