How do I stop the "Are you sure you want to open it?" dialog box from displaying when I open an app?
If you need to do this manually on a file (or files) you can run the following command to remove the Quarantine flag:

xattr -d com.apple.quarantine /PATH/TO/FILE(S)


So in your case since the quarantine flag is not being removed you will want to run the command with sudo as the primary cause for the flag not being removed after hitting "Open" is because you don't have permissions to do so.