MacOS High Sierra Messages app unread icon reappearing, causing system to hang
I think the solution can be found in DeerSteak's answer from: https://apple.stackexchange.com/a/332311/100302


Quit Messages.app
Open a Terminal and:

cd ~/Library/Messages/Archive

Search for the stuck message (compare timestamp) e.g.

find . -iname 'annoying person*.ichat'

Once you've located the offender, delete it
Restart the Dock

killall Dock