Is 1.6 seconds a typical time to fetch nearly 3.7 million rows from a table?
Both 1.6s and 7s seem reasonable.  Some issues that lead to variations:


Caching (Was I/O needed, or was everything sitting in RAM -- a factor of 10 variation is possible)
Distance to other server (Ping time)
Bandwidth to other server (Mb/sec)
Client's ability to absorb that much data that fast
Whether client is receiving it in chunks or all at once


But my question to you is -- what can you do with 3.7M rows all at once?  That is, this does not seem like a reasonable benchmark or whatever.

Back to the "caching".  Try this:


restart mysqld
run your test
run your test again -- likely to run faster.


Main cache:  for a 4GB server and InnoDB tables, innodb_buffer_pool_size should be something like 2G.