What makes CPU cache memory so much faster than main memory? I can see some benefit in a tiered cache system. It makes sense that a smaller cache is faster to search. But there must be more to it.
Why is CPU cache memory so fast?
memory;caching
In the case of a CPU cache, it is faster because it's on the same die as the processor.  In other words, the requested data doesn't have to be bussed over to the processor; it's already there.In the case of the cache on a hard drive, it's faster because it's in solid state memory, and not still on the rotating platters.In the case of the cache on a web site, it's faster because the data has already been retrieved from the database (which, in some cases, could be located anywhere in the world).So it's about locality, mostly.  Cache eliminates the data transfer step.Locality is a fancy way of saying data that is close together, either in time or space. Caching with a smaller, faster (but generally more expensive) memory works because typically a relatively small amount of the overall data is the data that is being accessed the most often.Further ReadingCache (Computing) on Wikipedia