MySQL select * from table without conditions does not return rows right away
What language are you using in your client?  What connector?  There may be a setting there.

Guessing that it is python, I see this:

https://dev.mysql.com/doc/connector-python/en/connector-python-api-cext-buffered.html

and guess that this would do the trick:

    cnx.buffered(True)


before the lengthy SELECT *.  It is not likely to help SELECT COUNT(*) since the is nothing to 'buffer' until it is finished.

Since you added details about using the Windows cli mysql.exe utility, See if mysql --quick achieves your goal.  If that doesn't provide the speed you require, perhaps consider using a loop, with a LIMIT clause, and "remember where you left off".  This would make each call to mysql.exe relatively quick, at the expense of having to call it many times, once for each window of rows.