1. A method for accessing a shared memory in a system having multiple application programming interfaces (APIs), the method comprising: registering a memory buffer that resides in a memory space and is accessible by at least two APIs of the multiple APIs, wherein a memory manager is configured to retain the memory buffer in the memory space to allow the at least two APIs to access the memory buffer while the memory buffer is registered and not move the memory buffer to a host memory space; requesting access to the memory buffer; requesting modifications to resource policies associated with the memory buffer to retain the memory buffer at a location accessible by the at least two APIs; synchronizing access to the memory buffer among the at least two APIs using a semaphore mechanism for purposes of accessing the memory buffer; and generating one or more calls that cause a processing unit to operate on data stored in the memory buffer.