What is the iMessage character limit?
(This answer doesn't apply to the latest version of Messages in 10.12 which appears capable of sending VERY large messages)

TL;DR; In 10.10.5 the limit is 18996 but you can't paste in a block that large.

This isn't the answer you want but it's the answer you should listen to:

iMessage isn't designed for sending huge long messages. Write an email instead.

But if you really want an answer the max characters is 18996 but the catch is that iMessage on OSX (10.10.5) won't let you paste in a single block of text that big. Try breaking up and pasting in smaller 1000 character chunks if you must use this platform to send large amounts of text.