How can I decide whether I can use a Raspberry Pi for my application?
Okay, this is more of an opinion than a typical definitive stackoverflow type answer, but here it goes.

I tried using a Raspberry Pi for image processing for my capstone project. Perhaps it was just the libraries, or having a bunch of stuff on USB, but I was extremely unsatisfied with its performance. It had a hard time streaming VGA images at any appreciable framerate, let alone doing any sort of image processing with them and then streaming it over.

I mean, there's a lot of variables (especially wrt code optimization and implementation), and I can't speak objectively and quantitatively, but you get the performance you would expect out of a computer you would only paid $30 for. I would suggest taking a look at Odroid X2 which has far more hardware kick, although I haven't used one myself. 

You could buy one and try it and see if it meets your demands; you'd be out only $30 if it didn't. But based on my own personal experience with this thing I really doubt this thing can do what you want at 120x a second.