1. A method for using speculative cache requests to reduce cache miss delays, the method comprising: receiving a memory reference which is directed to a cache line in a cache, wherein the cache is part of a multi-level memory hierarchy; and while determining whether the cache line is available in the cache, determining whether the memory reference is likely to miss in the cache, and if so, simultaneously sending a speculative request for the cache line to a lower level of the multi-level memory hierarchy, wherein determining whether the memory reference is likely to miss in the cache comprises using a program counter for an instruction for the memory reference to identify a counter in a set of counters that corresponds to the memory reference, wherein the counter counts at least one of hits and misses for the instruction for the memory reference separately from hits and misses for other instructions.