1. A parallel computing system for adding a plurality of first floating point numbers, the system comprising: a plurality of computing nodes, a computing node including at least one processor and at least one memory device; and a collective logic device including: a front-end logic device for receiving a plurality of the first floating point numbers in parallel from the computing nodes or network links and for converting the first floating point numbers to a plurality of integer numbers, said front-end logic device further for comparing exponents of each of the first floating point numbers to determine a maximum exponent; an ALU comprising a plurality of levels with combining blocks for adding the integer numbers in parallel and generating a summation of all of the plurality of integer numbers in one pass; and a back-end logic device for converting the summation to a second floating point number by performing shifting according to the maximum exponent, wherein the receiving, the converting the first floating point numbers, the adding, the generating and the converting the summation are performed in one pass in which the computing nodes send the plurality of first floating point numbers only once to the collective logic device and receive the second floating point number only once from the collective logic device.