OS X - How to have starting applications (optionally) not take the focus?
open -jga TextEdit opens TextEdit hidden (-j) and without making it frontmost (-g). The -j option was added in 10.8 and it is only shown in the help message and not in the man page. If TextEdit is already running, open -ja TextEdit (without -g) would just make TextEdit frontmost and not hidden. For some applications like TextEdit and Mail, open -jg opens a new visible window if the application is running but has no open windows, but you can use something like pgrep -x TextEdit||open -jga TextEdit as a workaround.

A second option in 10.9 and later is lsappinfo launch launchandhide=true nofront=true /Applications/TextEdit.app.

A third option is osascript -e 'launch app "TextEdit"'. Unlike the open and lsappinfo commands, it is blocked until the application finishes opening. For some applications like TextEdit and Mail, the launch command opens a new visible window if the application is not already running.

Edit: none of the options above seem to work with VirtualBox, but you can hide VirtualBox after opening it:

open -jga VirtualBox;lsappinfo setinfo -app VirtualBox kLSApplicationIsHiddenKey=true

osascript -e 'launch app "VirtualBox"' -e 'delay 1' -e 'tell app "System Events" to set visible of process "VirtualBox" to false'