Why/how does tile size effect performance?
The difference and why there are different optimal bucket sizes comes from the size and design of on chip cache/memory of cpu or gpu.


The gpu has a massive amount of cores. But they are dumb and cannot do much logic. They have a big data throughput though - so that's why is good to give them a big chunk of data. There will be a lot of parallelization done on that data and each core will do it's dumb thing over and over again till it's done. The parallelization happens internally in gpu, so in reality you would have to assign like 2x2 or 4x4 tile to each of those little cores, but you would have 3000 of those tiles. But since they share cache/vram, you assign a big tile to them. Therefore the more CUDA cores, the bigger tile. It's worth noting that Kepler, Maxwell or Pascall cuda cores are not equal, so it's best to test the ideal tile size between generations.

When mixing different gpus (if you must), it's generally best to go with the tile size of the least powerfull one. Many commercial renderers divide the last tile into smaller ones, so the cores/gpus that are finished can still render. This does not happen yet with Cycles, so it might happen that the last tile can end up as a job for the slowest gpu, and if the tile is too big, the rest of gpus will wait too long for the next frame doing nothing.
The cpu has low amount of cores, but comparably to gpu's cores they are super-powerful. Each such core can operate on a bigger chunk of data, and since individual cores do not share cache, you have that many tiles as you have cores (actually threads). Each core has approx 4-16MB of cache and that reflects the optimal tile size.