1. A method of performing collective communication in a collective communication system comprising processing nodes, the method comprising: determining whether a command message, regarding one function among a broadcast function, a scatter function, and a gather function, is generated by a processor; determining a transmission order between the processing nodes by assigning higher transmission orders to processing nodes that are not currently in communication and assigning lower transmission orders to processing nodes that are currently in communication, based on a status of each of the processing nodes if it is determined that the command message regarding the one function among the broadcast function, the scatter function, and the gather function, is generated by the processor; and performing communication with respect to the command message based on the determined transmission order.