Applescript Expected “"” but found unknown token
The spaces mess things up. Try:
do shell script &quot;open -n &quot; &amp; quoted form of &quot;/Applications/Game Capture HD.app&quot;

Essentially, 'quoted form' is for passing text to 'do shell script'. Both the Script Editor and the shell will be interpreting the text and 'quoted form' helps manage that.
For more information, refer to the Applescript Language Guide &gt; Class Reference &gt; Text &gt; Properties of text objects
Additional info can be found in Technical Note TN2065: do shell script in AppleScript