Suspected hard disk failure - Is it possible to boot from just a USB drive?
First off: Welcome to Ask Different.

Yes, you should be able to boot off of a USB drive in the way you describe.

It shouldn't be necessary to remove the HDD, though (to diagnose, that is). When you're turning on your Mac, you can hold down ⌥ (option/alt) to choose which device to boot from (Where you could select your USB drive). Booting from the recovery partition or a USB drive with the OS X installer on it, will give you access to both Terminal and Disk Utility.

Having dealt with disk failure in the past, I would advise you to try running AHT (Apple Hardware Test), if the Mac in question is from before June 2013, or Apple Diagnostics if it's newer. Click on the links for instructions, you won't need to start OS X to get to AHT/AD.

AHT/AD will be able to tell you if there's a hardware problem. If the HDD is truly busted, there's a chance AHT will stall after having run for 3-4 min (Happened to me) which in itself will "confirm" that the HDD isn't working like it should.

If you can't even get the computer to boot, obviously, you won't be able to start AHT/AD.