1. A semiconductor memory device comprising: a memory cell array a plurality of memory cells each of which stores N-bit data (N is an integer greater than 1); a peripheral circuit configured to perform a program operation to the memory cells in the memory cell array; and a control logic configured to control the peripheral circuit and the memory cell array such that, during a program operation to target memory cells to be programmed among the memory cells, a preprogram is performed to memory cells to be programmed to a highest program state among the target memory cells based on a preprogram reference voltage, and then a main program is performed to the target memory cells.