Speed up simple MySQL query on large database
Since you're reading every record in the table, the database will simply pull everything you want, wrap it [all] up and send it back across the network to your client machine.  The more data you pull, the larger that payload is going to be and the slower it's going to travel.

I frequently run this very simple query
The database is static and the data inside the database will not change.

To me, these two statements makes no sense together.
If the data does not change, then you should read it once, perhaps at application startup, and hold it in the client application.  Don't keep re-reading the same thing over and over again.  That's pointless.
Then there's the question of what you're doing with all this data after you've retrieved it from the database.  I sincerely hope you're not putting it all on a screen in front of a User - a million rows to read through?  No thanks!
I think you need to explain your situation a little more.