1. A semiconductor memory device, comprising: a memory cell array including a plurality of memory cell layers in a first direction, one of the memory cell layers having a plurality of first lines, a plurality of second lines intersecting the first lines, and a plurality of memory cells provided at intersections of the first lines and the second lines and operative to store data in accordance with different resistance states; and an access circuit operative to access to the memory cells via the first lines and the second lines, one of the memory cells changing a resistance state from a first resistance state to a second resistance state on application of a voltage of a first polarity, and changing the resistance state from the second resistance state to the first resistance state on application of a voltage of a second polarity opposite in polarity to the first polarity, the access circuit operative to, during accessing to selected one of the memory cells, set a first voltage to the first lines and the second lines, then set a second voltage higher than the first voltage to the first lines and the second lines, then set a third voltage higher than the second voltage to a first line or a second line connected to the selected memory cell, set the first voltage to the other one, and bring at least one of a first line and a second line connected to non-selected one of the memory cells into a state in which certain potential is not supplied/applied from external, certain one of the memory cell layers sharing the first lines with another memory cell layer located adjacent to the certain memory cell layer on one side in the first direction, and sharing the second lines with another memory cell layer located adjacent thereto on the other side, the access circuit including a sense amplifier operative to sense a cell current flowing in the selected memory cell via the first line, and a driver operative to supply a voltage, required for access to the selected memory cell, to the second line, and the driver shared among second lines disposed at odd-numbered/even-numbered orders counted in the first direction, of second lines disposed on the same position in the memory cell layers.