Why we need to complete circuit
The idea of a closed circuit works for low frequencies - where the corresponding wavelength is much larger than the components and wires.  Kirchoff's laws hold.

Things get tricky when the frequency is higher. A sudden change in voltage propagates at the speed of light (or some good fraction of it in cables, transmission lines) and there is more current at one point than at another.

In theory, you could put a sharp-edged voltage pulse on one lead of an LED, have nothing connected to the other, and for a tiny instant in time, as the pulse passes through the LED, have enough current for it to glow. But it would be extremely brief.   

So what if you send a series of pulses?  

A good rule of thumb to remember is at light speed, one nanosecond is about one foot (30+ cm).  LEDs and the pulse-pushing circuitry I imagine would be a few inches (or cm) and so things happen on a scale of maybe tenths of nanoseconds. You'd have to work with frequencies at several GHz.  

Another problem - every positive pulse you put on the anode lead will go through the LED and add positive charge to the non-connected cathode lead.  Each positive pulse will add more.  That charge has nowhere to escape to - just a tiny bit can flow back as leakage current, no diode being perfect.   From a physics point of view, so what?  Just let the whole contraption develop a positive charge.  Figure a few milliamps lasting for say 50 ns, times 5 billion times per second (just making up numbers), you quickly get to coulombs of charge, and many volts in just seconds.  At a practical level, it's not very practical at all.

I wonder if it would work better to have two LEDs wired anti-parallel, and feed GHz pulses to one end of the pair and leave the other end disconnected?   (I leave that thought for others to discuss.)