1. A method of erasing non-volatile storage, comprising: applying an erase voltage to a set of non-volatile storage elements while enabling erasing of each non-volatile storage element in said set, said set including a first subset of non-volatile storage elements and a second subset of non-volatile storage elements; verifying whether said set is erased after applying said erase voltage by testing conduction through said set while applying a verify voltage to said first subset of non-volatile storage elements and applying a first voltage to said second subset of non-volatile storage elements, said first voltage is different from said verify voltage; repeating said applying said erase voltage and verifying whether said set is erased until successfully verifying that said set is erased; and applying said erase voltage to said set while inhibiting erasing of said first subset after successfully verifying that said set is erased.