1. A signal processing device comprising: a signal processing unit which receives an input stream comprised of a plurality of pieces of input data, executes a predetermined task on stream data, and outputs an output stream comprised of a plurality of pieces of output data, wherein input data of the input stream is sequentially inputted to the signal processing unit, and output data configuring the output stream is sequentially outputted from the signal processing unit, wherein the signal processing device comprises a pointer indicating position information of data in the stream data according to progress of processing by the signal processing unit, wherein when priority processing of a second task is requested during execution of a first task, the signal processing device executes the second task after saving a value held by the pointer, wherein upon completion of the second task, based on the saved pointer value the signal processing device obtains position information, in the output stream, of output data to be outputted next in the first task, and obtains position information, in the input stream, of head input data of all input data that needs to be inputted to the signal processing unit to calculate the output data to be outputted next.