How to minimize sliding door gap
One thing you can do is to get some weather stripping and install it so that it just comes up to the glass slider. If the glass slider is one that free hangs without a track or guide at the bottom it will be somewhat problematic to have the weather stripping actually contact the glass as it will likely just push the bottom of the door way from the wall. 

Here is one style that you would mount to the inside edge of the door opening behind the slider.





Will it look ugly? Probably - but then almost anything you use to try to seal up that opening will not be pretty. Comment: Using that type of door for an environmental barrier is anything but optimum.