1. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to track virtual memory access by a graphics processing unit of a device, the method comprising: detecting a virtual memory access to a virtual memory address by a processing unit, wherein the device includes a central processing unit and the graphics processing unit that are each able to access the virtual memory address; determining if the processing unit is a graphic processing unit; and if the processing unit is the graphics processing unit, setting a graphics processing unit reference bit in a page table entry of a page table that corresponds to the virtual memory address, wherein the graphics processing unit reference bit indicates whether the graphics processing unit has recently accessed the virtual memory address, the page table entry further includes a central processing unit reference bit that indicates whether the central processing unit has recently accessed the virtual memory address, and the page table is shared between the central processing unit and the graphic processing unit.