iMessage unread count on Mac
Quit Messages
Open Activity Monitor, find the process called "Dock", then hit the top left "X" icon to force it the Dock process to quit. The dock should disappear and then reappear.
Reopen Messages. Your unread count should be reset!