1. A method of operating non-volatile storage, comprising: determining whether non-volatile storage elements in a group have a threshold voltage that is above or below a first voltage level; determining whether the non-volatile storage elements in the group have a threshold voltage that is above or below a second voltage level; determining a count of how many of the non-volatile storage elements in the group showed a different result between the first voltage level and the second voltage level; determining whether the count is greater than a threshold; correcting for misreads of the non-volatile storage elements in the group if the count is not less than the threshold; storing data from correcting the misreads into a set of data latches; and storing data from the non-volatile storage elements into the set of data latches without attempting to correct for misreads if the count is less than the threshold.