Using SFP modules from a microcontroller
Looking at what's in the SFP modules, it may not be all that difficult to use them directly. 

Since the signals are AC coupled to LVDS, you'll need to communicate in a DC balanced protocol such as Manchester, which your chosen ARM may or may not support (my current favorite Microchip SAME70 does support it). And, of course, add LVDS receiver/transmitters. You may be able to ignore the EEPROM since you only want to work with a single type of SFP module. 

Judging by the values shown, you'd probably have to keep the frequency at 10MHz or higher, which you will need to do anyway for that data rate.  

May or may not be worth looking into further, but it does look promising to me. Here's an excerpt from the above pdf document: