1. A method implemented on a computer for managing a parallel cache hierarchy in a processor, the method comprising: receiving an instruction that includes a cache operations modifier that identifies a first level of the parallel cache hierarchy in which to cache data associated with the instruction; selecting a first cache replacement policy from among a plurality of cache replacement policies, wherein the first cache replacement policy is selected based on whether the instruction comprises a load instruction or a store instruction; and executing the instruction and performing at least one related cache operation within the first level of the parallel cache hierarchy according to the first cache replacement policy.