Arduino USB Power vs External Power Supply
The design assumes USB provides a regulated 5v so no further regulation is necessary. The power connecter is meant to accept a wider range of voltage and regulate it to 5v, and the kind of regulators used on the Arduino board need that much higher voltage to provide regulation. If you already have a regulated +5v source you can supply it to the 5v pin. You'll bypass the on-board regulator this way, so just know that it will then be up to you to provide clean power.