Instances vs. cores when using EC2
A general rule of thumb is to not distribute until you have to. It's usually more efficient to have N servers of a certain capacity than 2N servers of half that capacity. More of the data access will be local, and therefore fast in memory versus slow across the network.

At a certain point, scaling up one machine becomes uneconomical because the cost of additional resource scales more than linearly. However this point is still amazingly high.

On Amazon in particular though, the economics of each instance type can vary a lot if you are using spot market instances. The default pricing more or less means that the same amount of resource costs about the same regardless of the instance type, that can vary a lot; large instances can be cheaper than small ones, or N small instances can be much cheaper than one large machine with equivalent resources.

One massive consideration here is that the computation paradigm can change quite a lot when you move from one machine to multiple machines. The tradeoffs that the communication overhead induce may force you to, for example, adopt a data-parallel paradigm to scale. That means a different choice of tools and algorithm. For example, SGD looks quite different in-memory and in Python than on MapReduce. So you would have to consider this before parallelizing.

You may choose to distribute work across a cluster, even if a single node and non-distributed paradigms work for you, for reliability. If a single node fails, you lose all of the computation; a distributed computation can potentially recover and complete just the part of the computation that was lost.