1. A convolution operation circuit comprising: a plurality of multipliers; a first shift register configured to provide data to a first input of the plurality of multipliers; a first storage unit configured to store a plurality of data to be supplied to the first shift register; a second shift register configured to provide data to a second input of the plurality of multipliers; a second storage unit configured to store a plurality of data to be supplied to the second shift register; a plurality of cumulative adders configured to accumulate output from the plurality of multipliers; a third storage unit configured to latch output from the plurality of cumulative adders at predetermined timing; a fourth storage unit configured to store data to be stored in the first storage unit and the second storage unit and data output from the third storage unit; and a control unit configured to control operations of the first storage unit, the second storage unit, the third storage unit, the fourth storage unit, the first shift register, the second register, and the cumulative adder, wherein the control unit sets a plurality of data stored in the first storage unit to the first shift register at predetermined timing, sets a plurality of data stored in the second storage unit to the second shift register at predetermined timing, causes the first shift register and the second shift register to perform shift operations in synchronization with an operation of the cumulative adder, and transfers during the shift operation data stored in the fourth storage unit to at least one of the first storage unit and the second storage unit.