Why is it required to have your MacBook Pro plugged-in in closed-clamshell mode with an external display attached?
Why? Why not allow or require it in both cases?

The default behavior when you close the lid on your MacBook is to put it to sleep.  According to Apple:
To further expand on this point, let's look at the default behavior when the lid is open(ed) - the MacBook is awake.  This makes sense as this is the generally accepted method of using a notebook.  The external display, and USB ports are also, by the inherent nature of the computer being awake, functional.

You can save energy by putting your Mac to sleep when you're not using
it. When your Mac is sleeping, it's on but consumes less power. And
waking your Mac from sleep is quicker than starting it up after it's
been shut down. To put your Mac to sleep immediately, do one of the
following:

Choose Apple () menu &gt; Sleep.
If you use a Mac notebook computer, close its built-in display.
Press Command (⌘)–Media Eject(⏏).
Tap the power button on your computer.


(Emphasis mine)
Having the AC adapter connected is just one of the components needed to override the default behavior; you need to have the AC power, keyboard and mouse connected, as well as a monitor. See Use your Mac notebook computer in closed-display mode with an external display for more details.
Why did Apple do it this way?  Ultimately, only they can give you the official answer, but a little deduction tells us that if closing the lid puts the machine to sleep, how would it know to stay awake if the user wanted to operate in closed-clamshell mode?  Answer:  attach a monitor and the AC adapter.
Can you override the default behavior
Yes.  Using pmset
To disable wake/sleep on lid state:
sudo pmset -a lidwake 0

To undo the last setting
sudo pmset -a lidwake 1

Though you can do this, I don't recommend it, because disabling this feature means when you close the lid on your laptop it remains awake; even when you pack it in your laptop bag and put it in the overhead compartment.  It (the battery) will most likely be DOA, plus it tends to get quite hot in that scenario, shortening the life of the battery.