Delay between electric current and light intensity of LED
How long is your fiber, what is the switching time of your emitter and where are you measuring?

You would get a 6ms delay on a 1800km long fiber due to the speed of light. 

Another source of delay is afterburn in the LED phosphor. If you are using LEDs designed for fiberoptic transmission they would typically have switching speeds down in the low ns range. There are slower/low cost ones with switching times up to 1us. I work with visible light LED for photography and they exhibit an afterburn in the hundreds of ns. It seems like delay in the LED itself is unlikely.

The speed of your photodetector could of course be a factor. The one I use has a 1 ns rise time and it is hard to imagine the detector itself causing a 6ms delay.

Where are you measuring? I would look at the circuitry that is driving the LED.