1. A parallel processing apparatus, comprising: a program counter for indicating instructions to be read out from a memory; a plurality of instruction registers for respectively storing instructions therein indicated by said program counter; a plurality of arithmetic units for executing arithmetic operations; means for controlling said plurality of arithmetic units to effect either parallel processing of a plurality of consecutive instructions read out from an address of said memory indicated by said program counter in said plurality of arithmetic units, or successive processing of n consecutive instructions, read out from an address of said memory indicated by said program counter, in which said n consecutive instructions correspond to said plurality of arithmetic units, respectively, when the instruction is supplied to each of the arithmetic units from the instruction register.