Is there a maximum lift to drag ratio?
With current technology the L/D might go up to 70 or 75, and going higher would require an almost impractically large wing span. Gliders need to fly in tight circles to use updrafts, and the larger the wingspan becomes, the bigger the speed difference between inner and outer wing will be. Also, landing such a wide wing without dropping a wingtip will be very hard. Smaller wings with a high aspect ratio will have a low chord length, leading to smaller Reynolds numbers, which translates into a steep increase of friction drag if the aspect ratio is increased without increasing wing area. Therefore, only adding wing span will help, and this runs into a soft wall beyond the 30 m of designs like the Eta. In addition, the current mass limit of 850 kg will make bigger aircraft unattractive for competition pilots.

The Concordia (pdf!) is claimed to have an L/D approaching 75, but I have learned over the years to see theoretical predictions as invariably optimistic, and realistic performance, with bugs on the wing and all, will never quite measure up to the hoped-for ideal. 

Werner Pfenniger proposed to use boundary layer laminarisation by suction (pdf!) to reduce friction drag and proposed glider designs with L/D ratios in excess of 100. Turbines at the wingtips would drive the suction pumps, so these would still be unpowered aircraft. But so far nobody has dared to turn his visions into reality.

While laminarisation avoids the increase in friction drag of a turbulent boundary layer, a moving wall would eliminate viscous losses altogether. Now the friction generation is between the moving wall (this could be a taut foil running between two cylinders at the forward and aft end of the wing) and the fixed structure. With current materials there is no hope of reducing drag this way, but who knows what tricks will be possible in the future.

A glide ratio of 100 or more looks rather unlikely in the next 50 years.