Regarding to the speed of supercomputer
The only thing you are missing is the importance of the interconnect between cores/processors/racks and the design of the memory system for each processor.

I assume you are referring to the most recent top-500 supercomputer list.  That list is sorted by Rmax, not Rpeak.

Rpeak is a measure of the theoretical maximum number of floating point operations that the system can perform.  For Rpeak you are correct: just add more cores (or more vector units per core) and the Rpeak number will go up.

Rmax is (arguably) a little bit more meaningful.  It is the maximal rate at which the system can run the Linpack benchmark.  The Linpack benchmark requires the system to solve a huge, dense, system of linear equations using LU factorization with partial pivoting in double precision.

For solving a dense system of linear equations you need to carefully partition the problem among the cores, and there needs to be an efficient way for the cores to communicate results back and forth.  So a big part of designing a computer that will get onto the Top500 list is careful design of the interconnect and the software used to send messages between processors.

On the top500 list they tell you what interconnect each system is using. Note that you don't find a system with Ethernet until you get to number 49.  The first 48 all have infiniband, cray gemini or custom interconnects.

There is also a green500 list that measures the MFLOPS/Watt of systems while solving the Linpack benchmark.  The difference between the top system and the 500th system on that list (from last November) is a factor of 75x.

Most people agree that there is a lot more that you want from a supercomputer in addition to being able to efficiently solve massive dense systems of linear equations.  Jack Dongara (who originated the Linpack benchmark) has recently released a paper proposing a new benchmark that measures the speed at which a system can solve a sparse linear system.  http://www.netlib.org/utk/people/JackDongarra/PAPERS/HPCG-Benchmark-utk.pdf.