hd video player for mac
The comments in the link you provide indicate the latest vlc-git nightly release should work with 10-bit mkv files.

On the VLC nightly build page, look for the first file with a file name that starts with "vlc-git-intel" or "vlc-git-intel64". Download the former if you want the 32-bit version, the latter for 64-bit. (Even though the vlc-git files don't indicate their version number in the file name, the latest nightly build, as of 10/6/11, is a version 1.2 build)

Be aware these are considered unstable versions, so you may come across unexpected bugs while using the software.

You could also try MplayerX, which I think should support 10-bit video.