In this paper, one new parallel version of GPBi-CG method (PGPBi-CG method, in brief) is proposed for solving large sparse linear systems with unsymmetrical coefficient matrices on distributed parallel environments. The method reduces three global synchronization points to one by reconstructing GPBi-CG method and the communication time required for the inner product can be efficiently overlapped with computation time of vector updates. It combines the elements of numerical stability with the characteristics of design of parallel algorithms. The cost is only slightly increased computation time, which can be ignored, compared with the reduction of communication time. Performance and isoefficiency analysis shows that the PGPBi-CG method has better parallelism and scalability than the GPBi-CG method. Numerical experiments show that the scalability can be improved by a factor 3 and the improvement in parallel communication performance approaches 66.7 %.