Persistent badge on Messages app icon in macOS High Sierra
None of the other answers really solved it for me. After killing the Dock or signing out and back in, I'd have the badge return eventually. 

So in Finder I went to ~/Library and searched for the listed phone number. It showed me a file in ~/Library/Messages/Archive/yyyy-mm-dd/ that was named for the phone number. Deleted the file and then killed the dock (by executing killall Dock in the Terminal) and the badge was gone instantly.

I have also figured out the cause: deleting a message on your iPhone or iPad without reading it. It's always an automated message with a 2FA code or something, and that's because I could see the code and typed it in and then deleted the thread later because I always trash that junk.