1. A cache management system comprising: a processor; a memory; a secondary storage device formatted according to a file system; a non-transitory computer readable storage medium storing a set of computer instructions, the computer instructions executable to: manage a multi-level cache comprising in-memory cache stored in the memory and file system based cache stored on the secondary storage device; store a file in the file system based cache on the secondary storage device as a cached file; store an in-memory version of the cached file in the memory; maintain an in-memory access count for the in-memory version of the cached file; encode a file system based cache access count for the cached file in an access count timestamp in file system metadata associated with the cached file; and determine whether to evict the cached file from the file system based cache based on the access count timestamp.