How does a GPU process a task by using multiple ALUs? I know the philosophy of how a GPU works {many weak men doing a task (GPU) compared to one strong man doing the same task (CPU)}. 

How does the GPU manage all the ALUs and schedule the necessary calculation to an ALU?
Architecture Background

GPU don't schedule to individual ALUs, but rather groups of them. Terminology will vary based on GPU company (NVidia, ATI, and Intel), but I will stick with NVidia terminology since they all work somewhat similarly at the high level.

NVidia has a concept of CUDA cores, each of which contains an ALU and/or FPU, which make up collections called Streaming Multiprocessors (SM). The Fermi architecture has 32 CUDA cores per SM, for example.

GPUs operate in a SIMD-like manner. This means that unlike a CPU, which traditionally has had one execution unit per instruction, GPUs have many execution units per instruction (CUDA core). These groups of execution are called warps.

Answer

GPUs contain scheduling units that attempt to reach maximal utilization. Warps are scheduled in a way to utilize as much of the GPU as possible. One way to do this is to use a round-robin approach. Latency is something that a GPU struggles with compared to a CPU, so scheduling a process as soon as possible is often desirable. Like an OS (which controls CPU scheduling), however, simplicity and hiding latency are still desirable traits to an extent, although other tradeoffs may be deemed more important.

Your workload will determine how scheduling is done. If you run a CUDA kernel or shader on 320 pieces of data (threads), you will need at least 10 warps to be scheduled (this depends on the type of data as well). If you have 1 thread that needs to be calculated, oh well, you still need to schedule one warp to do the calculation.

Of course, what ratchet freak said is true about GPU architectures and algorithms being proprietary, especially when compared to CPUs.