1. A method for selectively storing cachelines in a multi-level cache hierarchy comprising a plurality of cache memories, the method comprising: tracking reuse behavior of at least one cacheline, during execution of an application, in both a first level cache memory and a second level cache memory; the first level cache memory being smaller and faster than the second level cache memory; and determining a cache placement policy by tracking reuse behavior for the at least one cacheline in both the first level cache memory and the second level cache memory and determining the cache placement policy for both the first and the second level cache memories based on the reuse behavior, wherein the cache placement policy includes selectively placing the at least one cacheline in both the first level cache memory and the second level cache memory, wherein each cacheline is part of one of a plurality of datasets and further where the reuse behavior observed for the at least one cacheline is used to determine a cache placement policy for the plurality of cachelines being part of the same dataset as the at least one cacheline.