Why are cassettes replaced more often than front chainrings and derailleur jockey wheels?
To expand on Jahaziel's comment, and address your point that you have 5x as many cogs at the back (though anywhere from 3x to 11x is common):
You don't use all your gears equally.
Most pedalling on most bikes suited to their riding conditions is near the middle of the cassette, with the more extreme sprockets for the steepest climbs or fast descents/flats. You only need to wear out one commonly used sprocket to need a new cassette. In my case it's always the 3rd or 4th, counting from the biggest, depending on the bike, though if I get to the point where I can feel that, wear is very visible on some others. This will have fewer teeth than (most of) your chainring(s).
Tooth count has a double effect - fewer teeth in contact to spread the load, and each tooth encounters than chain more often.
On all my bikes, though probably least the MTB, the cassette gets dirtier than the chainrings - more abrasive grit.
I suspect a further effect of sprocket size beyond tooth count: a worn chain/sprocket combination that doesn't mesh well is likely to do more damage when the chain is bent more.
The jockey wheels last longer despite being small and plastic for two reasons: much less contact force against the chain to cause wear, and they don't have to be very good to do their job