1. A system to manage memory using access-prioritized garbage collection based on evaluating monitored-load instructions to the exclusion of unmonitored-load instructions, the system comprising: a processor; and a memory storing program code, which, when executed on the processor, performs an operation comprising: designating a first region and a second region within a memory as load-monitored regions; configuring a performance monitor to count object pointer accessed events of the first and second load-monitored regions, respectively; executing an unmonitored-load instruction to load data from the memory; executing a monitored-load instruction to load a pointer that points to a first location in the memory; responsive to determining that the first location is within at least one of the first and second load-monitored regions, triggering an object pointer accessed event, wherein no object pointer access event is triggered: (i) as a result of any unmonitored-load instruction; and (ii) as a result of any monitored-load instruction that loads a pointer not pointing to any location within the first and second load-monitored regions; responsive to triggering the object pointer accessed event, updating, in the performance monitor, at least one of: (i) a first count of object pointer accessed events in the first load-monitored region and (ii) a second count of object pointer accessed events in the second load-monitored region; and performing access-prioritized garbage collection including, upon determining that the first count exceeds the second count, prioritizing garbage collection of the first load-monitored region over the second load-monitored region such that garbage collection is performed on at least part of the second load-monitored region only after garbage collection is performed on at least part of the first load-monitored region.