How to open Mono GUI applications on OS X?
This might seem like a silly suggestion but are you giving the applications enough time to launch?

I tried installing and launching both applications. The first launch took a surprising amount of time; as you describe the icon appears in the Dock almost immediately but then no UI seems to appear. However if you leave it long enough the UI then does appear (about 4-5 minutes on a 2011 Macbook Pro).

Subsequent launches are a lot faster.

If you are curious to see what it is doing you can run this sudo opensnoop -v | grep mono-sgen in a Terminal when you launch the applications. On my system the delay on initial launch just seems to be Mono reading all the fonts on my system and caching them in ~/.cache/fontconfig. Luckily it just seems wants to do that once.