in mac os x, how NOT to bring all the opened window of an application to front
That's how Mac works.  

All windows belong to the same instance of the same app, unlike windows where it's possible to have each window inside a different process.

The simplest workaround is to duplicate the app you need 2 instances of &amp; drag the file you wish to launch to one specific copy of that app.

Alternatively, you might be able to do it in terminal or Applescript/shellscript, but you would need to pass the correct arguments to it on the fly...
do shell script "\"/Applications/myApp.app/Contents/MacOS/myApp\" + arguments [at least file path]

I'm not sure all apps will work with this method, I've only ever tried it with very specific structures with published command line switches.