Parallel and distributed computing What is(are) the difference(s) between parallel and distributed computing? When it comes to scalability and efficiency, it is very common to see solutions dealing with computations in clusters of machines, and sometimes it is referred to as a parallel processing, or as distributed processing.

In a certain way, the computation seems to be always parallel, since there are things running concurrently. But is the distributed computation simply related to the use of more than one machine, or are there any further specificities that distinguishes these two kinds of processing? Wouldn't it be redundant to say, for example, that a computation is parallel AND distributed?
Simply set, 'parallel' means running concurrently on distinct resources (CPUs), while 'distributed' means running across distinct computers, involving issues related to networks.

Parallel computing using for instance OpenMP is not distributed, while parallel computing with Message Passing is often distributed.

Being in a 'distributed but not parallel' setting would mean under-using resources so it is seldom encountered but it is conceptually possible.