Splitting the table to get more clusterization options - will it help?
Since this is a theoretical question I'm sure there will be corner cases where that is a good idea but as a general rule I wouldn't go down that path.

You will, as you state, complicate your regular queries by having to join both tables and you will also complicate your inserts/updates.

A well designed, properly normalized, database with proper indexing on a server that has been sized well should perform adequate with a couple of million rows (and a lot more than that).

For reference, the stack exchange network runs on a mainstream RDBMS and uses no special design tricks. You can verify that for yourself by downloading the database.