1. A multi-level hierarchical memory system, which includes at least one level of cache as a primary cache, comprising: a processor for processing requests for accessing data, the processor performing a method comprising: receiving a request for data in the multi-level hierarchical memory system, the request including a virtual address for accessing the data; querying a translation buffer to obtain an absolute address corresponding to the virtual address; responsive to the translation buffer not containing an absolute address corresponding to the virtual address, querying a translation unit to obtain the absolute address corresponding to the virtual address; performing a directory look-up upon obtaining the absolute address to determine whether a matching absolute address exists in a directory indicating that a line is available in a primary cache for accessing the requested data; sending a fetch request for the requested data to a next level in the multi-level hierarchical memory system above the primary cache, wherein processing of the fetch request by the next level in the multi-level hierarchical memory system occurs in parallel with the directory lookup; and receiving the requested data from the next level in the hierarchical memory system in the primary cache to make the requested data available to be written to the primary cache as needed, depending on the results of the directory lookup.