The L LED Keeps Itself On
This is a “feature” introduced in the revision 3 of the Uno board. Up to
Rev 2, the “L” LED was connected to pin 13, through a 1&nbsp;kΩ
resistor. On Rev 3, the same LED+resistor combination is driven by pin
13 through an op-amp. Thus, when pin 13 is in high impedance mode (i.e.
INPUT mode, the default mode when the program starts), the input of
the op-amp is floating. The voltage at this input (and thus at the
output) is not controlled and depends only on the parasitic conductances
and on leakage currents.

If you want to turn the LED off, you not only need to write LOW to it,
you also (and most importantly) should set the pin to OUTPUT.