Reference book for Parallel Computing and Parallel algorithms.
Also wanted to know that from which reference book or papers are the concepts in the udacity course on Parallel Computing taught...?


The History of Parallel Computing goes back far in the past, where the current interest in GPU computing was not yet predictable. Some important concepts date back to that time, with lots of theoretical activity between 1980 and 1990. Compared to the complicated trade-offs for current heterogenous systems, those old theoretical algorithms and concepts feel nice, powerful and easy to understand. Uzi Vishkin is a researcher who worked during that period, who still believes in those concepts (without overly compilicated trade-offs):


  Alas, the software spiral is now broken: (a) nobody is building hardware that provides improved performance on the old serial software base; (b) there is no broad parallel computing application software base for which hardware vendors are committed to improve performance; and (c) no agreed-upon architecture currently allows application programmers to build such software base for the future


I recently learned about those old concepts, and corresponding interesting survey articles:

Parallel Algorithms (64 pages)
https://www.cs.cmu.edu/~guyb/papers/BM04.pdf
by Guy E. Blelloch and Bruce M. Maggs – 1996

Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques (104 pages)
http://www.umiacs.umd.edu/users/vishkin/PUBLICATIONS/classnotes.pdf
by Uzi Vishkin – October 12, 2010

A Survey of Parallel Algorithms for Shared-Memory Machines (70 pages)
https://www2.eecs.berkeley.edu/Pubs/TechRpts/1988/5865.html
https://www2.eecs.berkeley.edu/Pubs/TechRpts/1988/CSD-88-408.pdf
by Richard M. Karp and Vijaya Ramachandran – March 1988

Limits to Parallel Computation: P-Completeness theory (327 pages)
https://homes.cs.washington.edu/~ruzzo/papers/limits.pdf
by R Greenlaw, H Hoover, W Ruzzo – 1995

Parallel Algorithms (347 pages)
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.466.8142&amp;rep=rep1&amp;type=pdf
by H Casanova, A Legrand, Y Robert – 2008