need help "hacking" LED display
My first idea is that hacking the software may require more work than building a display from scratch. You could cannibalize the display itself and connect your own controller to it.  

If you want to do it with the SM89516A, it appears to have a 8051 core, but on the pinout I can't see any special pins for programming. It probably uses a pair of I/O pins like UART to communicate with the programmer. There's no standard for this, I've already seen different protocols for it, which means you're out of luck if you can't find it documented in some SyncMOS document.
You may have to disconnect the programming pins because they may be used for other functions. This could mean patching the PCB as well.
I would go for a new controller. The interface will be simple if you use the existing display as a module.  

Looking at the documents on the SyncMOS website, they don't look very encouraging. Application note for programming is an animated GIF(!), showing how to program the device on a Hi-Lo programmer. Nothing wrong with Hi-Lo, but I tend to stay away as far as possible from companies which don't manage to create better documents.
The Hi-Lo also suggest that the programming protocol may be somewhat involved.