Performance difference between Text and Varchar in Mysql
The performance difference is mainly due to the fact, that text datatype columns are always copied to temporary tables on disk, because the memory engine doesn't support those types.

From the manual:


  Instances of BLOB or TEXT columns in the result of a query that is
  processed using a temporary table causes the server to use a table on
  disk rather than in memory because the MEMORY storage engine does not
  support those data types (see Section 8.4.4, “Internal Temporary Table
  Use in MySQL”). Use of disk incurs a performance penalty, so include
  BLOB or TEXT columns in the query result only if they are really
  needed.


When you don't use group by no temporary table is needed, obviously.