Attaining maximum lens speed requires engineering tradeoffs, and as such, "prime" (fixed focal length) lenses are generally faster than zoom lenses, and modern manual-focus lenses are generally faster than their autofocus counterparts [REF].
Especially outside of the "normal lenses", lens speed also tends to correlate with the price and/or quality of the lens.