Is there a Mac text editor that has live display of unique word count?
Emacs can do it (as most things). Have a look at this page. The part: "How many times was each word used?" - is quite similar to your problem. WordCountMode shows how to put the info in the modeline. Shouldn't be too hard to get in a form you need.