Fiber-optic transceiver ICs for applications other than fiber-optic Ethernet?
SONET is kind of popular in the telecomms world, very different to ethernet, and kind of cool in its way. 

In reality many non ethernet uses exist, but we tend to try to use rates that are close enough to something used by either the phone company or the datacenter because economies of scale make optical modules for those line rates all kinds of cheap. 

MADI for example (multichannel digital audio used in pro environments) often goes over 155Mb/s SFP modules marketed for SONET use by the telecom crowd, IIRC it actually runs the line at 125Mb/s, but close enough. 

SDI digital video is another example, either at 1.55Gb/s, 3Gb/s or 12Gb/s, again SFP modules that are close enough usually exist.

Particularly with SFP interfaces they usually don't care about the protocol as long as DC balance is not too horrible and whatever you are sending can survive being highpassed at a few MHz (The ALC will stuff up the levels at the the slicer if you go slower then that). Apart from that, generate a PECL version of whatever you like and stuff it into a SFP+ socket, odds are it will work if you pick a reasonable module for the speed you need.