Downside of manual shutdown
Yes, there are a number of downsides!!!

The reason this is faster is it's not as clean. When you choose "Shut Down" from the Apple menu, the OS sends an Apple Event to every running application asking it to quit. (Specifically, the quit Apple Event) If you have unsaved work, these apps will respond to this event by prompting you to save.

When you use /sbin/shutdown no apple events are sent! Instead, POSIX signals (specifically the TERM signal) are sent to every app. This tells the process to quit now, is likely to result in lost data because apps may not prompt you to save.

Using the "Shutdown" menu might present a dialog alerting you that certain applications have "canceled" the shutdown. This is a safety feature in case they have unsaved data. This will not happen with /sbin/shutdown. Any apps which don't respond to a TERM signal will be sent a KILL signal, which will forcibly terminate the app.

If you use "Legacy Filevault" (The version of Filevault where only your home directory is encrypted), then using the "Shutdown" command will initiate the process which frees up unused disk space in your "sparse bundle", the files which store the encrypted data for your home directory. /sbin/shutdown will not trigger this process.

There can be other unforeseen consequences as well. The outgoing firewall Little Snitch used to lose the firewall configuration when using /sbin/shutdown... they may have corrected this in later versions, but I know it was an issue last time I checked.

In summary, /sbin/shutdown will shut down the sore of the OS: it will flush disk buffers, and request (the POSIX way) that processes terminate, so you're not likely to experience disk corruption or serious side effects from using it. But it will not perform higher-level functions (like sending Apple Events) and therefore it is not the preferred way to shut down an OS X system. (But it's still far better than just killing the power)