How to know if the system console is open?
Short Answer: You cannot

First, bpy.ops.wm.console_toggle() is available only on Windows. And I admit, it is a work around. Other OSs run without a console. If you want a console, you must start blender from a terminal on Linux and Mac.

The I/O steam redirect blender.exe &gt; out.txt should work. (but I haven't test it)

If you realy want to communicate with user, you should use a dialog box:
http://wiki.blender.org/index.php/Dev:2.5/Py/Scripts/Cookbook/Code_snippets/Interface

error message or save it in a log file. Those methods should work on any system.

P.S.

On C code side, the function returns the current status, and you can query the status.