1. A parallel data processing method based on multiple graphic processing units (GPUs), comprising: creating, in a central processing unit (CPU), a plurality of worker threads for controlling a plurality of worker groups respectively, the worker groups comprising a plurality of GPUs; binding each worker thread to a corresponding GPU; loading one batch of training data from a nonvolatile memory to a GPU video memory corresponding to one worker group; transmitting, between a plurality of GPUs corresponding to one worker group, data required by data processing performed by the GPUs through peer to peer; and controlling the plurality of GPUs to perform data processing in parallel through the worker threads; the method further comprising: opening up write cache and read cache in a random access memory (RAM), sizes of the write cache and the read cache being the size of a storage structure configured to store one batch of training data*the total number of worker groups; processing all the worker threads in a barrier state before the write cache is full; and exchanging preset indexes pointing to the write cache and the read cache after the write cache is full.