1. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to access an object stored in a cache of a distributed storage system, the method comprising: determining if the object is in the cache that stores a plurality of objects, wherein the distributed storage system includes the cache and a distributed object layer for persistently storing the plurality of objects in each of the plurality of storage controller servers of the distributed storage system, the cache is a multi-layer cache and further includes a de-duplicated layer 1 and 2 cache, a granularity of allocation for the layer 1 and 2 caches is different, and writes to the layer 2 cache are done at a segment granularity and reads from the layer 2 cache are done at the object granularity; if the object is in the cache, accessing the object from the cache; and if the object is not in the cache, accessing the object from the distributed object layer.