1. A method for allowing an accelerator device to share physical memory of a computer system that is managed by and operates under control of an operating system of the computer system, the method comprising: creating a plurality of operating system (OS) page tables for memory management; creating, at a driver, a plurality of non-shared page tables maintained by the driver, wherein the non-shared page tables are independent of the operating system, are distinct from the OS page tables, and are to be used exclusively by the accelerator device to share the physical memory; pinning memory pages referenced in the plurality of non-shared page tables to prevent reuse of the memory pages referenced in the non-shared page tables; maintaining a global list of and a global counter for pinned memory pages for processes that interact with the accelerator device; and maintaining local lists of and local counters for pinned memory pages for respective processes that interact with the accelerator device.