OS X ML 10.8.2 sound from certain applications going to non-selected sound card?
Not entirely sure and I'm not familiar with iTerm2 but, as a workaround, does iTerm2 have audio preferences of its own?
If so, check to see if your USB device is available to select as the output device.

Also check Audio MIDI Setup (Applications > Utilities).