1. A method of crawling a hierarchical storage structure of a data storage system, the method comprising: recursively dividing the hierarchical storage structure into a plurality of sections, wherein the hierarchical storage structure contains data entities stored by the storage system; maintaining a queue in the data storage system, the queue containing leads for the plurality of sections to be crawled for metadata; associating each of the leads with one of a plurality of threads associated with a parallel-processing architecture of the data storage system; parallel-processing the plurality of sections utilizing the plurality of threads to generate a plurality of sorted lists of metadata, wherein each of the plurality of sorted lists corresponds to a different one of the plurality of sections of the hierarchical storage structure, said parallel-processing the plurality of sections including: identifying, by each thread, metadata associated with data entities contained in a section associated with the thread; appending, by the thread, the metadata of each of the data entities as a metadata entry to a metadata list corresponding to the associated section, the metadata list stored in a memory buffer associated with the thread; and sorting, by the thread, the metadata entries within the metadata list of the associated section based on a unique identifier associated with the metadata entry; merging, by the threads associated with each of the plurality of sections, corresponding sorted lists of metadata to form a baseline list, wherein the baseline list contains sorted metadata for the data entities of the hierarchical storage structure; and outputting a representation of the baseline list, as a result of the crawling, to indicate a state of the data entities stored by the storage system.