1. A method of programming non-volatile storage, comprising: selecting a first set of non-volatile storage elements; starting a program operation for the first set of non-volatile storage elements and completing the program operation for the first set of non-volatile storage elements; after starting the program operation for the first set of non-volatile storage elements and without waiting for completion of the program operation for the first set of non-volatile storage elements, unselecting the first set of non-volatile storage elements and selecting a second set of non-volatile storage elements; and starting a program operation for the second set of non-volatile storage elements without waiting for completion of the program operation for the first set of non-volatile storage elements.