1. A method for execution by a processor configured to operate with a dispersed storage network (DSN), the method comprises: identifying an encoded data slice of a set of encoded data slices that requires rebuilding; identifying storage units of the DSN that store the set of encoded data slices; determining a rebuilding metric for the identified storage units that store the set of encoded data slices by determining a response attribute that pertains to encoded data slice retrieval from the identified storage units to rebuild the identified encoded data slice; selecting a sub-set of the storage units for retrieving a decode threshold number of encoded data slices of the set of encoded data slices based on a performance metric associated with the response attribute for respective storage units that store the set of encoded data slices, wherein a data segment of data was error encoded to produce the set of encoded data slices, wherein the decode threshold number of encoded data slices is required to recover the data segment, and wherein the decode threshold number of encoded data slices does not include the identified encoded data slice; when the decode threshold number of encoded data slices has not been retrieved within a given time period: modifying the rebuilding metric; and selecting a new sub-set of the storage units for retrieving the decode threshold number of encoded data slices based on the modified rebuilding metric; when the decode threshold number of encoded data slices has been retrieved, decoding the decode threshold number of encoded data slices to produce a reconstructed data segment; and generating a rebuilt encoded data slice from the reconstructed data segment.