1. An apparatus comprising: a map structure stored in a non-volatile memory (NVM) as a plurality of map pages that associate logical addresses of user data blocks with physical addresses in the NVM at which the user blocks are stored; a first cache which stores a copy of a first subset of the map pages stored in the NVM; a second cache which stores a copy of the first subset of the map pages as well as a copy of a second subset of the map pages stored in the NVM; a programmable processor having programming stored in an associated memory which, when executed, issues a find command to locate and place a requested map page associated with a received host command into the first cache; and a non-programmable hardware circuit configured to, responsive to the find command, direct a retrieval of the requested map page from the NVM, access a reverse table in a memory to identify a potential replacement map page stored at a selected physical address in the second cache, and direct a writing of a copy of the requested map page to the selected physical address in the second cache responsive to a determination by the non-programmable hardware circuit that the potential replacement map page is not in the first subset of the map pages in the first cache.