1. An arithmetic processing apparatus performing arithmetic by a neural network in which a plurality of processing layers are hierarchically connected, the arithmetic processing apparatus, which corresponds to one of the plurality of processing layers comprising: a convolution arithmetic portion including at least two systolic arrays implemented as hardware, a flip flop circuit, and a weight coefficient memory, in which the convolution arithmetic portion receives an image data from a preceding layer of the processing layers, executes convolution arithmetic with respect to the image data received, and outputs, in each arithmetic cycle, a part of convolution arithmetic result data required for a pooling processing at one time, the at least two systolic arrays include a first systolic array and a second systolic array each having a first arithmetic line in which a plurality of product sum operators are connected in series, a second arithmetic line in which a plurality of product sum operators are connected in series, and a cumulative adder cumulatively adding arithmetic result data obtained from each of the first arithmetic line and the second arithmetic line, the at least two systolic arrays outputting a part of the convolution arithmetic result data which corresponds to an arrangement in a horizontal direction or a perpendicular direction of the image data, the first arithmetic line of the first systolic array, the first arithmetic line of the second systolic array, the second arithmetic line of the first systolic array and the second arithmetic line of the second systolic array are arranged in an order, the flip flop circuit inputs data received from the preceding layer while adjusting timing, each of the product sum operators includes a multiplier that multiplies the image data from the flip flop circuit and a weight coefficient from the weight coefficient memory and an adder that adds multiplication result, the weight coefficient memory provides, in common, a product sum operator of the first arithmetic line of the first systolic array and a product sum operator of the first arithmetic line of the second systolic array with the weight coefficient, wherein the first systolic array and the second systolic array share the weight coefficient in the weight coefficient memory, the flip flop circuit provides, in common, a product sum operator of the second arithmetic line of the first systolic array and a product sum operator of the first arithmetic line of the second systolic array with the input data, wherein the first systolic array and the second systolic array share the input data with timing adjusted by the flip flop circuit, whereby the input data and the weight coefficient are shared by the first systolic array and the second systolic array, each of the product sum operators of the at least two systolic array outputs the convolution arithmetic result data; and a pooling processing portion including a first comparator, a memory circuit, and a second comparator, the first comparator and the second comparator being implemented as hardware, and performs the pooling processing on the convolution arithmetic result data output by the convolution arithmetic portion as a result of a plurality of arithmetic cycles, before executing activation processing, in the pooling processing: the first comparator receives two convolution arithmetic result data outputted from the product sum operators of the at least two systolic arrays, outputs data having a larger value of the two convolution arithmetic result data as n-th max convolution arithmetic result data in a n-th arithmetic cycle to the memory circuit, and outputs data having a larger value of the two convolution arithmetic result data as a n+1-th max convolution arithmetic result data in a n+1-th arithmetic cycle to the second comparator, and the second comparator compares the n+1-th max arithmetic result data outputted from the first comparator at the n+1-th arithmetic cycle with the n-th max arithmetic convolution arithmetic result data stored in the memory circuit so as to output data having a larger value of the n+1-th max arithmetic result data and the n-th max arithmetic convolution arithmetic result data as a pooling arithmetic result data; and an activation processing circuit that activates the pooling arithmetic result data outputted by the second comparator, thereby the first comparator and the second comparator of the pooling processing provide the pooling arithmetic result data from the first and second systolic arrays, before activation processing by one of the activation processing circuit.