Use SFP modules for HDMI or Dp TMDS line
HDMI, besides the TDMS lines - which certainly could be bridged over optical - has several sideband signals, notably DDC (I2C) bus.
HDMI needs DDC to not only identify the monitor characteristics (VESA data), but more importantly, to authenticate the receiving device and manage encryption (HDCP) keying. That alone might prevent you from making it work.