fastest way of doing on/off-modulation of a LED?
If you are trying to send data this way, don't try to modulate it 0%-100%.
Go 10%-90%, this will be way faster.

To switch it off rapidly, you need 2 transistors in push-pull configuration, PNP+NPN or N-MOSFET + P-MOSFET, so that in 'off' state LED will be shorted to ground. Achieving high speed with BJT would be easier.

If you need to go over 1-5Mhz, you will need to add anti-saturation Schottky diodes.

Another thing to try is bridge circuit out of 4 BJT's - it will eliminate remaining charge in LED even faster (as LED will be reverse biased in off state), but I haven't tried that. Some LED's might die if reverse biased too much.