Why does my PostgreSQL query cost so much when I tack on an additional group-by?
The reason why the third statement is slow is it has to group 2.8 million rows which cannot be done in memory any longer. So the sorting is done on disk which obviously is a lot slower:

You can see that in this line:

Sort Method: external merge  Disk: 286688kB"


The number of rows and the time it took can be seen in this line:: 

Sort  (...) (actual time=19110.027..22635.209 rows=2807146 loops=1)

You can also see that the first and second statements did the grouping using a hash function and both only had to process less than 2000 rows.

HashAggregate  (cost=...) (actual time=1617.381..1617.799 rows=1794 loops=1)

The memory that Postgres is allowed to use for sorting is controlled through the configuration parameter work_mem. If you increase that until the sorting can be done in memory, the performance should get better.